It is possible to use Squid together with a firewall to secure internal networks from the outside using a proxy cache. The firewall denies all clients access to external services except Squid. All Web connections must be established by the proxy. With this configuration, Squid completely controls Web access.

If the firewall configuration includes a DMZ, the proxy should operate within this zone. Section 26.5, “Configuring a Transparent Proxy” describes how to implement a transparent proxy. This simplifies the configuration of the clients, because in this case they do not need any information about the proxy.

Several instances of Squid can be configured to exchange objects between them. This reduces the total system load and increases the chances of finding an object already existing in the local network. It is also possible to configure cache hierarchies, so a cache can forward object requests to sibling caches or to a parent cache—causing it to get objects from another cache in the local network or directly from the source.

Choosing the appropriate topology for the cache hierarchy is very important, because it is not desirable to increase the overall traffic on the network. For a very large network, it would make sense to configure a proxy server for every subnet and connect them to a parent proxy, which in turn is connected to the proxy cache of the ISP.

All this communication is handled by ICP (Internet cache protocol) running on top of the UDP protocol. Data transfers between caches are handled using HTTP (hypertext transmission protocol) based on TCP.

To find the most appropriate server from which to get the objects, one cache sends an ICP request to all sibling proxies. These answer the requests via ICP responses with a HIT code if the object was detected or a MISS if it was not. If multiple HIT responses were found, the proxy server decides from which server to download, depending on factors such as which cache sent the fastest answer or which one is closer. If no satisfactory responses are received, the request is sent to the parent cache.

Tip: Avoiding Duplication of Objects

To avoid duplication of objects in different caches in the network, other ICP protocols are used, such as CARP (cache array routing protocol) or HTCP (hypertext cache protocol). The more objects maintained in the network, the greater the possibility of finding the desired one.

Not all objects available in the network are static. There are a lot of dynamically generated CGI pages, visitor counters, and encrypted SSL content documents. Objects like this are not cached because they change each time they are accessed.

The question remains as to how long all the other objects stored in the cache should stay there. To determine this, all objects in the cache are assigned one of various possible states. Web and proxy servers find out the status of an object by adding headers to these objects, such as “Last modified” or “Expires” and the corresponding date. Other headers specifying that objects must not be cached are used as well.

Objects in the cache are normally replaced, because of a lack of free hard disk space, using algorithms such as LRU (last recently used). This means that the proxy expunges the objects that have not been requested for the longest time.

Speed plays an important role in the caching process, so this factor deserves special attention. For hard disks, this parameter is described as random seek time, measured in milliseconds. Because the data blocks that Squid reads from or writes to the hard disk tend to be rather small, the seek time of the hard disk is more important than its data throughput. For the purposes of a proxy, hard disks with high rotation speeds are probably the better choice, because they allow the read-write head to be positioned in the required spot more quickly. One possibility to speed up the system is to use a number of disks concurrently or to employ striping RAID arrays.

In a small cache, the probability of a HIT (finding the requested object already located there) is small, because the cache is easily filled and the less requested objects are replaced by newer ones. If, for example, one GB is available for the cache and the users only surf ten MB per day, it would take more than one hundred days to fill the cache.

The easiest way to determine the needed cache size is to consider the maximum transfer rate of the connection. With a 1 Mbit/s connection, the maximum transfer rate is 125 KB/s. If all this traffic ends up in the cache, in one hour it would add up to 450 MB and, assuming that all this traffic is generated in only eight working hours, it would reach 3.6 GB in one day. Because the connection is normally not used to its upper volume limit, it can be assumed that the total data volume handled by the cache is approximately 2 GB. This is why 2 GB of disk space is required in the example for Squid to keep one day's worth of browsed data cached.

The amount of memory (RAM) required by Squid directly correlates to the number of objects in the cache. Squid also stores cache object references and frequently requested objects in the main memory to speed up retrieval of this data. Random access memory is much faster than a hard disk.

In addition to that, there is other data that Squid needs to keep in memory, such as a table with all the IP addresses handled, an exact domain name cache, the most frequently requested objects, access control lists, buffers, and more.

It is very important to have sufficient memory for the Squid process, because system performance is dramatically reduced if it must be swapped to disk. The cachemgr.cgi tool can be used for the cache memory management. This tool is introduced in Section 26.6, “cachemgr.cgi”.

Squid is not a program that requires intensive CPU usage. The load of the processor is only increased while the contents of the cache are loaded or checked. Using a multiprocessor machine does not increase the performance of the system. To increase efficiency, it is better to buy faster disks or add more memory.

To start Squid, enter systemctl start squid at the command line as root. In the initial start-up, the directory structure of the cache must first be defined in /var/cache/squid. This is done automatically by the start script and can take a few seconds or even minutes. If done appears to the right in green, Squid has been successfully loaded. To test the functionality of Squid on the local system, enter localhost as the proxy and 3128 as the port in the browser.

To allow users from the local system and other systems to access Squid and the Internet, change the entry in the configuration files /etc/squid/squid.conf from http_access deny all to http_access allow all. However, in doing so, consider that Squid is made completely accessible to anyone by this action. Therefore, define ACLs that control access to the proxy. More information about this is available in Section 26.4.2, “Options for Access Controls”.

After modifying the configuration file /etc/squid/squid.conf, Squid must reload the configuration file. Do this with systemctl reload squid. Alternatively, completely restart Squid with systemctl restart squid.

The command systemctl status squid can be used to check if the proxy is running. The command systemctl stop squid causes Squid to shut down. This can take a while, because Squid waits up to half a minute (shutdown_lifetime option in /etc/squid/squid.conf) before dropping the connections to the clients and writing its data to the disk.

Warning: Terminating Squid

Terminating Squid with kill or killall can damage the cache. To be able to restart Squid, a damaged cache must be deleted.

If Squid dies after a short period of time even though it was started successfully, check whether there is a faulty name server entry or whether the /etc/resolv.conf file is missing. Squid logs the cause of a start-up failure in the file /var/log/squid/cache.log. If Squid should be loaded automatically when the system boots, enable the service with systemctl enable squid.

An uninstall of Squid does not remove the cache hierarchy or the log files. To remove these, delete the /var/cache/squid directory manually.

Setting up a local DNS server makes sense even if it does not manage its own domain. It then simply acts as a caching-only name server and is also able to resolve DNS requests via the root name servers without requiring any special configuration (see Section 19.4, “Starting the BIND Name Server”). How this can be done depends on whether you chose dynamic DNS during the configuration of the Internet connection.

Tip: DNS and Firewall

If you have a firewall running, make sure DNS requests can pass it.

Squid provides a detailed system for controlling the access to the proxy. By implementing ACLs, it can be configured easily and comprehensively. This involves lists with rules that are processed sequentially. ACLs must be defined before they can be used. Some default ACLs, such as all and localhost, already exist. However, the mere definition of an ACL does not mean that it is actually applied. This only happens with http_access rules.

To inform squid that it should act as a transparent proxy, use the option transparent at the tag http_port in the main configuration file /etc/squid/squid.conf. After restarting squid, the only other thing that must be done is to reconfigure the firewall to redirect the HTTP port to the port given in http_port. In the following squid configuration line, this would be the port 3128.

http_port 3128 transparent

Now redirect all incoming requests via the firewall with help of a port forwarding rule to the Squid port. To do this, use the enclosed tool SuSEFirewall2, described in Book “Security Guide”, Chapter 15 “Masquerading and Firewalls”, Section 15.4.1 “Configuring the Firewall with YaST”. Its configuration file can be found in /etc/sysconfig/SuSEfirewall2. The configuration file consists of well-documented entries. To set a transparent proxy, you must configure several firewall options:

Define ports and services (see /etc/services) on the firewall that are accessed from untrusted (external) networks such as the Internet. In this example, only Web services are offered to the outside:

FW_SERVICES_EXT_TCP="www"

Define ports or services (see /etc/services) on the firewall that are accessed from the secure (internal) network, both via TCP and UDP:

FW_SERVICES_INT_TCP="domain www 3128"
FW_SERVICES_INT_UDP="domain"

This allows accessing Web services and Squid (whose default port is 3128). The service “domain” stands for DNS (domain name service). This service is commonly used. Otherwise, simply take it out of the above entries and set the following option to no:

FW_SERVICE_DNS="yes"

The most important option is option number 15:

# 15.)
# Which accesses to services should be redirected to a local port on
# the firewall machine?
#
# This option can be used to force all internal users to surf via
# your squid proxy, or transparently redirect incoming webtraffic to
# a secure webserver.
#
# Format: 
# list of <source network>[,<destination network>,<protocol>[,dport[:lport]]
# Where protocol is either tcp or udp. dport is the original
# destination port and lport the port on the local machine to
# redirect the traffic to
#
# An exclamation mark in front of source or destination network
# means everything EXCEPT the specified network
#
# Example: "10.0.0.0/8,0/0,tcp,80,3128 0/0,172.20.1.1,tcp,80,8080"

The comments above show the syntax to follow. First, enter the IP address and the netmask of the internal networks accessing the proxy firewall. Second, enter the IP address and the netmask to which these clients send their requests. In the case of Web browsers, specify the networks 0/0, a wild card that means “to everywhere.” After that, enter the original port to which these requests are sent and, finally, the port to which all these requests are redirected. Because Squid supports protocols other than HTTP, redirect requests from other ports to the proxy, such as FTP (port 21), HTTPS, or SSL (port 443). In this example, Web services (port 80) are redirected to the proxy port (port 3128). If there are more networks or services to add, they must be separated by a blank space in the respective entry.

FW_REDIRECT="192.168.0.0/16,0/0,tcp,80,3128"

To start the firewall and the new configuration with it, change an entry in the /etc/sysconfig/SuSEfirewall2 file. The entry START_FW must be set to "yes".

Start Squid as shown in Section 26.3, “Starting Squid”. To verify that everything is working properly, check the Squid log files in /var/log/squid/access.log. To verify that all ports are correctly configured, perform a port scan on the machine from any computer outside your network. Only the Web services (port 80) should be open. To scan the ports with nmap, the command syntax is nmap -O IP_address.

First, a running Web server on your system is required. Configure Apache as described in Chapter 24, The Apache HTTP Server. To check if Apache is already running, as root enter the command systemctl status apache2. Otherwise, enter systemctl start apache2 to start Apache with the openSUSE Leap default settings. The last step to set it up is to copy the file cachemgr.cgi to the Apache directory cgi-bin. For 32-bit, this works as follows:

cp /usr/lib/squid/cachemgr.cgi /srv/www/cgi-bin/

In a 64-bit environment, the file cachemgr.cgi is located below /usr/lib64/squid/ and the command to copy it to the Apache directory is the following:

cp /usr/lib64/squid/cachemgr.cgi /srv/www/cgi-bin/

There are some default settings in the original file required for the cache manager. First, two ACLs are defined, then http_access options use these ACLs to grant access from the CGI script to Squid. The first ACL is the most important, because the cache manager tries to communicate with Squid over the cache_object protocol.

acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255

The following rules give Apache the access rights to Squid:

http_access allow manager localhost
http_access deny manager

These rules assume that the Web server and Squid are running on the same machine. If the communication between the cache manager and Squid originates at the Web server on another computer, include an extra ACL as in Example 26.2, “Access Rules”.

acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl webserver src 192.168.1.7/255.255.255.255 # webserver IP

Then add the rules in Example 26.3, “Access Rules” to permit access from the Web server.

http_access allow manager localhost
http_access allow manager webserver
http_access deny manager

Configure a password for the manager for access to more options, like closing the cache remotely or viewing more information about the cache. For this, configure the entry cachemgr_passwd with a password for the manager and the list of options to view. This list appears as a part of the entry comments in /etc/squid/squid.conf.

Restart Squid every time the configuration file is changed. Do this easily with systemctl reload squid.

Go to the corresponding Web site—http://webserver.example.org/cgi-bin/cachemgr.cgi. Press continue and browse through the different statistics.