For wiking with moin on nginx, I tried a lot and finally got it work. I’m using Moin 1.7.2 with Nginx 0.7.19 on Slackware linux.

As you may know, there is a mod called mod_wsgi which is an implementation of the Python Web Server Gateway Interface for the nginx web server. However, mod_wsgi code is about 7 months old and only tested with nginx 0.5.x, currently cannot be built with nginx 0.7.x. I didn’t test it with nginx 0.6.x, but there are some patches. For more information, check http://hg.mperillo.ath.cx/nginx/mod_wsgi/file/tip/README (CANNOT access from m^therf^cking China mainland network).

I intended to use the public wiki mode, but unfortunately, haven’t work it out on fastcgi. Any help is welcome. :-)

It was Cliff who illumined me that nginx can proxy to moin standalone server. Thanks to Cliff.

After some test, here is a block of the working nginx configuration on moinmoin wiki.

...snip...
    server {
	listen       127.0.0.1:80;
	server_name	localhost;
	server_tokens	off;
	root	/srv;

# Begin Core MoinMoin:
	rewrite	^/moin_static[0-9]*/(.*)$	/moin/$1	last;
	rewrite ^/(favicon\.ico)$		/moin/$1	last;

### temporary use
	location = / {
		rewrite .*	/wiki/	redirect;
	}
###

	location /wiki {
		proxy_redirect		http://localhost:8000	/;
		proxy_pass		http://localhost:8000;
		proxy_set_header	X-Real-IP	$remote_addr;
		proxy_set_header	X-Forwarded-For	$proxy_add_x_forwarded_for;
		proxy_set_header	Host		$http_host;
		proxy_set_header        X-Moin-Location /wiki/;
	}
# End Core MoinMoin.

	location  ~* \.(jpg|jpeg|gif|png|ico|css|js|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|mov|avi|mkv|srt|idx|sub)$ {
	    access_log	off;
	    expires	30d;
	}

	location ~* /\.ht {
            deny  all;
	}
    }
...snip...

Then, http://localhost/wiki is ready.
It’s an important point of the X-Moin-Location header. Moin will add that path to the output link.
So far, so good.

Today Willy Tarreau gave me some advise. So I took a short time re-ran this benchmark. Here follows the detail.

platform
Everything kept unchanged except for the proxy_server, I replaced it with this new one:

    o OS: Linux 2.6.21.5-smp i686 (Slackware 12.0)
    o CPU:  Intel(R) Xeon(TM) CPU 3.06GHz x 2
    o MEM: 1024M x 6
    o DISK: RAID 5
    o Ethernet controller:  Intel Corporation 82546EB Gigabit Ethernet Controller

file pool
This time, I generated 2 sets of 10Mbyte files, one is 1,000 files of 10kbyte size, and the other is 10 files of 1MByte size.

benchmark

    * client: http_load
    * proxy server: varnish 1.1.2, squid 2.6.STABLE18, and squid 3.0.STABLE2.
    * http server: nginx/0.6.28

Using the same configurations of nginx, varnish, and squid 2/3, got the results below:

+++ 10KByte +++

$ http_load -verbose -parallel 100 -fetches 100000 ./10k.urls

100000 fetches, 100 max parallel, 1.024e+09 bytes, in 14.7505 seconds
10240 mean bytes/connection
6779.43 fetches/sec, 6.94213e+07 bytes/sec
msecs/connect: 0.400918 mean, 11.452 max, 0.067 min
msecs/first-response: 14.0161 mean, 1779.32 max, 0.24 min
HTTP response codes:
code 200 — 100000

* Squid 2.6.STABLE18:

100000 fetches, 100 max parallel, 1.024e+09 bytes, in 26.1771 seconds
10240 mean bytes/connection
3820.13 fetches/sec, 3.91181e+07 bytes/sec
msecs/connect: 0.497665 mean, 2990.79 max, 0.055 min
msecs/first-response: 21.0663 mean, 3018.84 max, 4.071 min
HTTP response codes:
code 200 — 100000

* Squid 3.0.STABLE2:

— 60.0027 secs, 100000 fetches started, 96249 completed, 0 current
100000 fetches, 100 max parallel, 9.85651e+08 bytes, in 102.375 seconds
9856.51 mean bytes/connection
976.8 fetches/sec, 9.62785e+06 bytes/sec
msecs/connect: 2.56114 mean, 91.428 max, 0.061 min
msecs/first-response: 27.8048 mean, 94.563 max, 1.288 min
3751 timeouts
3745 bad byte counts
HTTP response codes:
code 200 — 96255

+++ 1MByte +++

$ http_load -verbose -parallel 100 -fetches 1000 ./1m.urls

* Varnish 1.1.2:

— 60 secs, 6640 fetches started, 6540 completed, 100 current
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 91.1719 seconds
1.04858e+06 mean bytes/connection
109.683 fetches/sec, 1.15011e+08 bytes/sec
msecs/connect: 36.9187 mean, 9019.28 max, 0.08 min
msecs/first-response: 26.7986 mean, 475.462 max, 18.781 min
HTTP response codes:
code 200 — 10000

* Squid 2.6.STABLE18:

— 60 secs, 5856 fetches started, 5756 completed, 100 current
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 103.829 seconds
1.04858e+06 mean bytes/connection
96.3126 fetches/sec, 1.00991e+08 bytes/sec
msecs/connect: 2.4862 mean, 2994.65 max, 0.063 min
msecs/first-response: 15.9743 mean, 134.817 max, 8.222 min
HTTP response codes:
code 200 — 10000

* Squid 3.0.STABLE2:

— 60 secs, 6083 fetches started, 5983 completed, 100 current
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 103.054 seconds
1.04858e+06 mean bytes/connection
97.0367 fetches/sec, 1.0175e+08 bytes/sec
msecs/connect: 6.6513 mean, 3022.23 max, 0.089 min
msecs/first-response: 16.0642 mean, 787.308 max, 0.741 min
HTTP response codes:
code 200 — 10000

proxy server system status

+++ 10KByte +++

10k-varnish-iostat_xm5
10k-squid2-iostat_xm5
10k-squid3-iostat_xm5
10k-varnish-vmstat5
10k-squid2-vmstat5
10k-squid3-vmstat5

+++ 1MByte +++

1m-varnish-iostat_xm5
1m-squid2-iostat_xm5
1m-squid3-iostat_xm5
1m-varnish-vmstat5
1m-squid2-vmstat5
1m-squid3-vmstat5

proxy status

)-: Hmm, I’m still not satisfied with these results, especially the something about squid3.
Maybe I need to optimize the configuration of the proxies?

I just use the DNS forward(cache) function here, which only serves with the systems of internal network, as the configuration file of the dnsmasq server below:

$ cat /etc/dnsmasq.conf

no-hosts
neg-ttl=3600
#log-queries
log-facility=/var/log/dnsmasq.log
pid-file=/var/run/dnsmasq.pid
user=nobody
group=nogroup
port=53
edns-packet-max=1280
interface=eth0 # internal network
except-interface=eth1 # external network
#no-dhcp-interface=eth1
listen-address=192.168.0.10
bind-interfaces
resolv-file=/etc/dnsmasq.resolv.conf
strict-order
#all-servers
stop-dns-rebind
#no-poll
cache-size=2048
no-negcache

$ cat /etc/resolv.conf

nameserver 127.0.0.1

On another system:

$ cat /etc/resolv.conf

nameserver 192.168.0.10
nameserver 192.168.0.11

I think it’s a good choice to effect fault tolerance with setting up two dnsmasq servers. As the two lines above.

the first time:

$ dig dotimes.com

;; Query time: 258 msec
;; SERVER: 192.168.0.10#53(192.168.0.10)

the second time:

$ dig dotimes.com

;; Query time: 1 msec
;; SERVER: 192.168.0.10#53(192.168.0.10)

As you can see the difference in the above example.
Yes, this is only one request, but what about surfing, or sending mails on mail server? It’s really worth the effort.

djbdns, pdnsd are also good DNS cache (proxy) programs, which yep worth a try.
The only thing I’m wondering that whether djbdns is still under maintainence.

I really got impressed with its stability, scalability and high performance. It doesn’t cost any system resource except memory usage, which depends on the connection number. And yet it’s very low-cost, for each connection entry consumes about 128bytes or so. It’s easy for you to figure it out that 100,000 unique connections/s, only 12.8M memory will be used. On the other hand, as a 4th-layer implement, IPVS just direct packages.

So, a question raised, what if I need layer-7 switching?

Yeah, IPVS no longer sucks. Here goes HAProxy, the reliable, high performance TCP/HTTP load balancer. Although there are other solutions, e.g., F5, pound, XLB, pen, and so on, I would like to chose HAProxy and introduce it here.

HAProxy is written by Willy TARREAU, the new Linux Kernel 2.4 maintainer since August 2006, who also maintains patches against the 2.6.20 branch since August of 2007.

Today I glanced my eye over the HAProxy Architecture Guide. I’d like to take a note here.

This note is so annoyed that you would better go for a drink, then YouTube. Yeah, I’m serious. :-) Otherwise, it’s better to check the link in the line above for original detail if you are interested in the architecture. Okay, here I go, see you…

1. Simple HTTP load-balancing with cookie insertion

  192.168.1.1    192.168.1.11-192.168.1.14   192.168.1.2
 -------+-----------+-----+-----+-----+--------+----
        |           |     |     |     |       _|_db
     +--+--+      +-+-+ +-+-+ +-+-+ +-+-+    (___)
     | LB1 |      | A | | B | | C | | D |    (___)
     +-----+      +---+ +---+ +---+ +---+    (___)
     haproxy        4 cheap web servers

Flows :

(client)                           (haproxy)                         (server A)
  >-- GET /URI1 HTTP/1.0 ------------> |
               ( no cookie, haproxy forwards in load-balancing mode. )
                                       | >-- GET /URI1 HTTP/1.0 ---------->
                                       | <-- HTTP/1.0 200 OK -------------<
               ( the proxy now adds the server cookie in return )
  <-- HTTP/1.0 200 OK ---------------< |
      Set-Cookie: SERVERID=A           |
  >-- GET /URI2 HTTP/1.0 ------------> |
      Cookie: SERVERID=A               |
      ( the proxy sees the cookie. it forwards to server A and deletes it )
                                       | >-- GET /URI2 HTTP/1.0 ---------->
                                       | <-- HTTP/1.0 200 OK -------------<
   ( the proxy does not add the cookie in return because the client knows it )
  <-- HTTP/1.0 200 OK ---------------< |
  >-- GET /URI3 HTTP/1.0 ------------> |
      Cookie: SERVERID=A               |
                                    ( ... )

If clients use keep-alive (HTTP/1.1), only the first response will have a cookie inserted, and only the first request of each session will be analyzed. The added server cookie will not be removed from the requests forwarded to the servers, so the server must not be sensitive to unknown cookies. If this causes trouble, keep-alive can be disabled by adding the following option :

option httpclose

If for some reason the clients cannot learn more than one cookie, and the application already produces a cookie, “prefix” mode could be used(see below).

Backing LB1 up using keepalived for healthcheck & failover(see below)

If the application needs to log the original client’s IP, use the “forwardfor” option which will add an “X-Forwarded-For” header with the original client’s IP address. Also use “httpclose” to rewrite every requests and not only the first one of each session :

option httpclose
option forwardfor

The web server will have to be configured to use this header instead. For example, on apache:

LogFormat “%{X-Forwarded-For}i %l %u %t \”%r\” %>s %b ” combined
CustomLog /var/log/httpd/access_log combined

In the situation of clients disable cookies on their browser, use the “source” balancing algorithm instead of the “roundrobin”. So a given IP address will always reaches the same server. (as long as the number of servers remains unchanged.) Never use this behind a proxy or in a small network, because the distribution will be unfair. However, in large internal networks, and on the internet, it works quite well. Clients
which have a dynamic address will not be affected as long as they accept the cookie, because the cookie always has precedence over load balancing.

2. HTTP load-balancing with cookie prefixing and high availability

Backed load-balancer up with a second one in VRRP mode using keepalived.

http://www.keepalived.org/

Allow the proxy to bind to the shared IP:

# echo 1 >/proc/sys/net/ipv4/ip_nonlocal_bind

    shared IP=192.168.1.1
  192.168.1.3  192.168.1.4    192.168.1.11-192.168.1.14   192.168.1.2
 -------+------------+-----------+-----+-----+-----+--------+----
        |            |           |     |     |     |       _|_db
     +--+--+      +--+--+      +-+-+ +-+-+ +-+-+ +-+-+    (___)
     | LB1 |      | LB2 |      | A | | B | | C | | D |    (___)
     +-----+      +-----+      +---+ +---+ +---+ +---+    (___)
     haproxy      haproxy        4 cheap web servers
     keepalived   keepalived

Set the “httpclose” option to disable keep-alive (HTTP/1.1), so the proxy can access to all cookies in all requests for each session, because the proxy will modify EVERY cookie sent by the client and the server.

Flows :

(client)                           (haproxy)                         (server A)
  >-- GET /URI1 HTTP/1.0 ------------> |
               ( no cookie, haproxy forwards in load-balancing mode. )
                                       | >-- GET /URI1 HTTP/1.0 ---------->
                                       |     X-Forwarded-For: 10.1.2.3
                                       | <-- HTTP/1.0 200 OK -------------<
                        ( no cookie, nothing changed )
  <-- HTTP/1.0 200 OK ---------------< |
  >-- GET /URI2 HTTP/1.0 ------------> |
    ( no cookie, haproxy forwards in lb mode, possibly to another server. )
                                       | >-- GET /URI2 HTTP/1.0 ---------->
                                       |     X-Forwarded-For: 10.1.2.3
                                       | <-- HTTP/1.0 200 OK -------------<
                                       |     Set-Cookie: JSESSIONID=123
    ( the cookie is identified, it will be prefixed with the server name )
  <-- HTTP/1.0 200 OK ---------------< |
      Set-Cookie: JSESSIONID=A~123     |
  >-- GET /URI3 HTTP/1.0 ------------> |
      Cookie: JSESSIONID=A~123         |
       ( the proxy sees the cookie, removes the server name and forwards
          to server A which sees the same cookie as it previously sent )
                                       | >-- GET /URI3 HTTP/1.0 ---------->
                                       |     Cookie: JSESSIONID=123
                                       |     X-Forwarded-For: 10.1.2.3
                                       | <-- HTTP/1.0 200 OK -------------<
                        ( no cookie, nothing changed )
  <-- HTTP/1.0 200 OK ---------------< |
                                    ( ... )

Setting “weight”(values between 1 and 256) to inform haproxy to spread the load of backends the most smoothly possible respecting those ratios:

server webA 192.168.1.11:80 cookie A weight 12 check
server webC 192.168.1.13:80 cookie C weight 26 check

2.1 Variations involving external layer 4 load-balancers

Can the haproxies be load-balanced?
Yeah, by a layer4 load-balancer (eg: Alteon) which will check the services:

              | VIP=192.168.1.1
         +----+----+
         | Alteon  |
         +----+----+
              |
 192.168.1.3  |  192.168.1.4  192.168.1.11-192.168.1.14   192.168.1.2
 -------+-----+------+-----------+-----+-----+-----+--------+----
        |            |           |     |     |     |       _|_db
     +--+--+      +--+--+      +-+-+ +-+-+ +-+-+ +-+-+    (___)
     | LB1 |      | LB2 |      | A | | B | | C | | D |    (___)
     +-----+      +-----+      +---+ +---+ +---+ +---+    (___)
     haproxy      haproxy        4 cheap web servers

What if the Alteon fails? Want relay generic TCP protocols (SMTP, TSE, VNC, etc…)? (see below)

2.2 Generic TCP relaying and external layer 4 load-balancers

Using the “monitor-net” keyword to specify a network which will be dedicated to monitoring systems and must not lead to a forwarding connection nor to any log. This expects a version of haproxy greater than or equal to 1.1.32 or 1.2.6.

                |  VIP=172.16.1.1   |
           +----+----+         +----+----+
           | Alteon1 |         | Alteon2 |
           +----+----+         +----+----+
 192.168.1.252  |  GW=192.168.1.254 |  192.168.1.253
                |                   |
          ------+---+------------+--+-----------------> TSE farm : 192.168.1.10
       192.168.1.1  |            | 192.168.1.2
                 +--+--+      +--+--+
                 | LB1 |      | LB2 |
                 +-----+      +-----+
                 haproxy      haproxy

3. Simple HTTP/HTTPS load-balancing with cookie insertion

  192.168.1.1    192.168.1.11-192.168.1.14   192.168.1.2
 -------+-----------+-----+-----+-----+--------+----
        |           |     |     |     |       _|_db
     +--+--+      +-+-+ +-+-+ +-+-+ +-+-+    (___)
     | LB1 |      | A | | B | | C | | D |    (___)
     +-----+      +---+ +---+ +---+ +---+    (___)
     apache         4 cheap web servers
     mod_ssl
     haproxy

Do not cache inserted cookies for security measures.

If the cookie works in “prefix” mode, there is no need to add the “nocache” option because it is an application cookie which will be modified, and the application flags will be preserved.

If apache 1.3 is used as a front-end before haproxy, it always disables HTTP keep-alive on the back-end, so the “httpclose” is needn’t.

To log client’s IP, configure apache to set the X-Forwarded-For header not on haproxy.

Flows :

(apache)                           (haproxy)                         (server A)
  >-- GET /URI1 HTTP/1.0 ------------> |
               ( no cookie, haproxy forwards in load-balancing mode. )
                                       | >-- GET /URI1 HTTP/1.0 ---------->
                                       | <-- HTTP/1.0 200 OK -------------<
               ( the proxy now adds the server cookie in return )
  <-- HTTP/1.0 200 OK ---------------< |
      Set-Cookie: SERVERID=A           |
      Cache-Control: private           |
  >-- GET /URI2 HTTP/1.0 ------------> |
      Cookie: SERVERID=A               |
      ( the proxy sees the cookie. it forwards to server A and deletes it )
                                       | >-- GET /URI2 HTTP/1.0 ---------->
                                       | <-- HTTP/1.0 200 OK -------------<
   ( the proxy does not add the cookie in return because the client knows it )
  <-- HTTP/1.0 200 OK ---------------< |
  >-- GET /URI3 HTTP/1.0 ------------> |
      Cookie: SERVERID=A               |
                                    ( ... )

What if only SSL is required and cache is not needed? (see below)

3.1. Alternate solution using Stunnel

Stunnel is a cheaper solution than Apache+mod_ssl. It doesn’t process HTTP or add X-Forwarded-For header by default. (there is a patch on the official haproxy site to provide this feature to recent stunnel versions.)

Stunnel will only process HTTPS. Haproxy will get all HTTP traffic, so add the X-Forwarded-For header for HTTP traffic in haproxy, but not for HTTPS traffic since stunnel will already have done it.

Use the “except” keyword to tell haproxy that connections from local host already have a valid header.

  192.168.1.1    192.168.1.11-192.168.1.14   192.168.1.2
 -------+-----------+-----+-----+-----+--------+----
        |           |     |     |     |       _|_db
     +--+--+      +-+-+ +-+-+ +-+-+ +-+-+    (___)
     | LB1 |      | A | | B | | C | | D |    (___)
     +-----+      +---+ +---+ +---+ +---+    (___)
     stunnel        4 cheap web servers
     haproxy

Description :
- stunnel on LB1 will receive clients requests on port 443
- it forwards them to haproxy bound to port 80
- haproxy will receive HTTP client requests on port 80 and decrypted SSL
requests from Stunnel on the same port.
- stunnel will add the X-Forwarded-For header
- haproxy will add the X-Forwarded-For header for everyone except the local
address (stunnel).

4. Soft-stop for application maintenance
4.1 Soft-stop using a file on the servers

Put a file on the server which will be checked by the proxy. Remove this file so that the proxy will treat this server as dead, and won’t send any new sessions, only old ones if the “persist” option is used. Wait a bit then stop the server when there isn’t http connection anymore. And then it’s time to do backend server maintenance.

This solution will effect the clients, not so good.

4.2 Soft-stop using backup servers

Set two different names to one server checked on different port(one is 80), they share the exact same cookies. Those servers will only be used when no other server is available for the same cookie.

When the web servers are started, only one named server is seen as available. On the web server, redirect the other different port to local port 80(e.g., use iptables).

When need maintenance, simply stop the server from responding on port 81 so that its standard instance will be seen as failed, but the other will still work. This won’t effect the clients.

4.2.1 Variations for operating systems without any firewall software

Beside the iptables solution above, this redirection can also be handled by a simple haproxy in tcp mode :

global
daemon
quiet
pidfile /var/run/haproxy-checks.pid
listen 0.0.0.0:81
mode tcp
dispatch 127.0.0.1:80
contimeout 1000
clitimeout 10000
srvtimeout 10000

Starting an haproxy instance with this configuration to start the web service, and killing this instance will make the port 81 stopping responding so that the web service could be stopped.

4.2.2 Centralizing the server management

It’s also an solution to do the port redirection on the load-balancer.

Another solution is to use the “COMAFILE” patch provided by Alexander Lazic, which is available for download here :

http://w.ods.org/tools/haproxy/contrib/

4.3 Hot reconfiguration

Send a SIGTTOU signal to the proxy and it will release the ports so that a new instance can be started.

If the new instance fails to start, sending a SIGTTIN signal back to the original processes will restore the listening ports.

Otherwise, sending a SIGUSR1 signal to the old one and it will exit after its last session ends.

If the old process still exists, sending a SIGTERM to the old process.

5. Multi-site load-balancing with local preference
5.1 Network diagram

Note : offices 1 and 2 are on the same continent as site 1, while office 3 is on the same continent as site 3. Each production site can reach the second one either through the WAN or through a dedicated link.

        Office1         Office2                          Office3
         users           users                            users
192.168  # # #   192.168 # # #                            # # #
.1.0/24  | | |   .2.0/24 | | |             192.168.3.0/24 | | |
  --+----+-+-+-   --+----+-+-+-                   ---+----+-+-+-
    |      | .1     |      | .1                      |      | .1
    |    +-+-+      |    +-+-+                       |    +-+-+
    |    |OP1|      |    |OP2|                       |    |OP3|  ...
  ,-:-.  +---+    ,-:-.  +---+                     ,-:-.  +---+
 (  X  )         (  X  )                          (  X  )
  `-:-'           `-:-'             ,---.          `-:-'
  --+---------------+------+----~~~(  X  )~~~~-------+---------+-
                           |        `---'                      |
                           |                                   |
                 +---+   ,-:-.                       +---+   ,-:-.
                 |SD1|  (  X  )                      |SD2|  (  X  )
   ( SITE 1 )    +-+-+   `-:-'         ( SITE 2 )    +-+-+   `-:-'
                   |.1     |                           |.1     |
   10.1.1.0/24     |       |     ,---. 10.2.1.0/24     |       |
        -+-+-+-+-+-+-+-----+-+--(  X  )------+-+-+-+-+-+-+-----+-+--
         | | | | |   |       |   `---'       | | | | |   |       |
      ...# # # # #   |.11    |.12         ...# # # # #   |.11    |.12
          Site 1   +-+--+  +-+--+              Site 2  +-+--+  +-+--+
          Local    |S1L1|  |S1L2|              Local   |S2L1|  |S2L2|
          users    +-+--+  +--+-+              users   +-+--+  +--+-+
                     |        |	                         |        |
   10.1.2.0/24    -+-+-+--+--++--      10.2.2.0/24    -+-+-+--+--++--
                   |.1       |.4                       |.1       |.4
                 +-+-+     +-+-+                     +-+-+     +-+-+
                 |W11| ~~~ |W14|                     |W21| ~~~ |W24|
                 +---+     +---+                     +---+     +---+
              4 application servers               4 application servers
                    on site 1                           on site 2

5.2 Description
5.2.1 Local users

- Office 1 users connect to OP1 = 192.168.1.1
- Office 2 users connect to OP2 = 192.168.2.1
- Office 3 users connect to OP3 = 192.168.3.1
- Site 1 users connect to SD1 = 10.1.1.1
- Site 2 users connect to SD2 = 10.2.1.1

5.2.2 Office proxies

- Office 1 connects to site 1 by default and uses site 2 as a backup.
- Office 2 connects to site 1 by default and uses site 2 as a backup.
- Office 3 connects to site 2 by default and uses site 1 as a backup.

6. Source balancing

Sometimes it may reveal useful to access servers from a pool of IP addresses instead of only one or two. Some equipments (NAT firewalls, load-balancers) are sensible to source address, and often need many sources to distribute the load evenly amongst their internal hash buckets.

7. Managing high loads on application servers

Limiting the number of connections between the clients and the servers. Setting haproxy to limit the number of connections on a per-server basis. It will then fill all the servers up to the configured connection limit, and will put the remaining connections in a queue, waiting for a connection to be released on a server.

So that,

* all clients can be served whatever their number without crashing the servers, the only impact it that the response time can be delayed.

* the servers can be used at full throttle without the risk of stalling, and fine tuning can lead to optimal performance.

* response times can be reduced by making the servers work below the congestion point, effectively leading to shorter response times even under moderate loads.

* no domino effect when a server goes down or starts up. Requests will be queued more or less, always respecting servers limits.

* it’s easy to achieve high performance even on memory-limited hardware. Indeed, heavy frameworks often consume huge amounts of RAM and not always all the CPU available. In case of wrong sizing, reducing the number of concurrent connections will protect against memory shortages while still ensuring optimal CPU usage.

Today, I finished a benchmark to compare the caching performance and status between Varnish and Squid, which get widely focused on as reverse proxies.
Here we go.

platform

The test-network is made up of:

    * D-Link 1024R, a 24-port Gigabit Switch
    * http_server:
          o OS: Linux 2.6.21.5-smp i686 (Slackware 12.0)
          o CPU: Intel(R) Xeon(TM) CPU 2.80GHz x 2
          o MEM: 1024M x 6
          o DISK: SEAGATE ST373405LC SCSI Disk
          o Ethernet controller: Intel 82546EB PRO/1000 MT Dual Port Server Adapter
    * proxy_server:
          o OS: Linux 2.6.21.5-smp i686 (Slackware 12.0)
          o CPU:  Intel(R) Pentium(R) III CPU family 1133MHz GenuineIntel
          o MEM: 1024M
          o DISK: SEAGATE ST318406LC SCSI Disk
          o Ethernet controller:  Intel 82557/8/9 PRO/100+ Server Adapter
    * client
          o OS: Linux 2.4.31 i686 (Slackware 10.2)
          o CPU: Intel(R) Xeon(TM) CPU 2.80GHz x 2
          o MEM: 1024M x 6
          o Ethernet controller: Intel 82546EB Gigabit Ethernet Controller

file pool

#!/bin/sh
#
# Written for generating 2 sets of 100Mbyte files,
# one is 1,000 files of 100kbyte size, and
# the other is 10 files of 10MByte size.
# by Cherife Li
#

docroot=/home/wwwroot/

# generate the first set
mkdir -p $docroot/100k
cd $docroot/100k
for i in `seq 1 10`; do
mkdir -p files-$i;
for j in `seq 1 100`; do
dd if=/dev/zero of=files-$i/$j bs=100K count=1 2> /dev/null;
done;
done

# generate the second set
mkdir -p $docroot/10m
cd $docroot/10m
for i in `seq 1 5`; do
mkdir -p files-$i;
for j in `seq 1 2`; do
dd if=/dev/zero of=files-$i/$j bs=10M count=1 2> /dev/null;
done;
done

$ cd /home/wwwroot/
$ find ./100k/ | grep 'files.*/.' | sed 's#./#http://benchmark.lo/#' > 100k.urls
$ find ./10m/ | grep 'files.*/.' | sed 's#./#http://benchmark.lo/#' > 10m.urls

benchmark

 ______        ____________        ___________
|      |--A-->|            |--B-->|           |
|client|      |proxy server|      |http server|
|______|<--D--|____________|<--C--|___________|

    * client: http_load
    * proxy server: varnish 1.1.2, squid 2.6.STABLE18, and squid 3.0.STABLE2.
    * http server: nginx/0.6.28

The Nginx http server only ran the stand alone http service.
The proxy server ran the proxy service only the one been benchmarking at a time.

http_load
It’s a good load-generator as it

    * allows random fetches from a list of URLs
    * allows a large number of parallel requests
    * is portable.

There are also other test tools, check this page for detail.

+++ 100KByte +++

$ http_load -verbose -parallel 100 -fetches 100000 ./100k.urls

* Varnish 1.1.2:

— 60 secs, 6868 fetches started, 6768 completed, 100 current
— 120 secs, 13720 fetches started, 13620 completed, 100 current
— 180 secs, 20570 fetches started, 20470 completed, 100 current
— 240 secs, 27432 fetches started, 27332 completed, 100 current
— 300 secs, 34285 fetches started, 34185 completed, 100 current
— 360 secs, 41140 fetches started, 41040 completed, 100 current
— 420 secs, 47996 fetches started, 47896 completed, 100 current
— 480 secs, 54854 fetches started, 54754 completed, 100 current
— 540 secs, 61709 fetches started, 61609 completed, 100 current
— 600 secs, 68565 fetches started, 68465 completed, 100 current
— 660 secs, 75419 fetches started, 75319 completed, 100 current
— 720 secs, 82279 fetches started, 82179 completed, 100 current
— 780 secs, 89131 fetches started, 89031 completed, 100 current
— 840 secs, 95989 fetches started, 95889 completed, 100 current
100000 fetches, 100 max parallel, 1.024e+10 bytes, in 876.794 seconds
102400 mean bytes/connection
114.052 fetches/sec, 1.16789e+07 bytes/sec
msecs/connect: 116.758 mean, 9111.16 max, 0.256 min
msecs/first-response: 120.03 mean, 2494.66 max, 0.614 min
HTTP response codes:
code 200 — 100000

* Squid 2.6.STABLE18:

— 60 secs, 6898 fetches started, 6798 completed, 100 current
— 120 secs, 13748 fetches started, 13648 completed, 100 current
— 180 secs, 20595 fetches started, 20495 completed, 100 current
— 240 secs, 27439 fetches started, 27339 completed, 100 current
— 300 secs, 34287 fetches started, 34187 completed, 100 current
— 360 secs, 41136 fetches started, 41036 completed, 100 current
— 420 secs, 47983 fetches started, 47883 completed, 100 current
— 480 secs, 54829 fetches started, 54729 completed, 100 current
— 540 secs, 61675 fetches started, 61575 completed, 100 current
— 600 secs, 68523 fetches started, 68423 completed, 100 current
— 660 secs, 75371 fetches started, 75271 completed, 100 current
— 720 secs, 82221 fetches started, 82121 completed, 100 current
— 780 secs, 89065 fetches started, 88965 completed, 100 current
— 840 secs, 95909 fetches started, 95809 completed, 100 current
100000 fetches, 100 max parallel, 1.024e+10 bytes, in 878.411 seconds
102400 mean bytes/connection
113.842 fetches/sec, 1.16574e+07 bytes/sec
msecs/connect: 116.02 mean, 9114.65 max, 0.224 min
msecs/first-response: 115.596 mean, 619.381 max, 0.86 min
HTTP response codes:
code 200 — 100000

* Squid 3.0.STABLE2:

— 60 secs, 6885 fetches started, 6785 completed, 100 current
— 120 secs, 13720 fetches started, 13620 completed, 100 current
— 180 secs, 20577 fetches started, 20477 completed, 100 current
— 240 secs, 27418 fetches started, 27318 completed, 100 current
— 300 secs, 34266 fetches started, 34166 completed, 100 current
— 360 secs, 41111 fetches started, 41011 completed, 100 current
— 420 secs, 47957 fetches started, 47857 completed, 100 current
— 480 secs, 54812 fetches started, 54712 completed, 100 current
— 540 secs, 61656 fetches started, 61556 completed, 100 current
— 600 secs, 68503 fetches started, 68403 completed, 100 current
— 660 secs, 75346 fetches started, 75246 completed, 100 current
— 720 secs, 82198 fetches started, 82098 completed, 100 current
— 780 secs, 89040 fetches started, 88940 completed, 100 current
— 840 secs, 95891 fetches started, 95791 completed, 100 current
100000 fetches, 100 max parallel, 1.024e+10 bytes, in 876.658 seconds
102400 mean bytes/connection
114.07 fetches/sec, 1.16807e+07 bytes/sec
msecs/connect: 115.858 mean, 9111.16 max, 0.26 min
msecs/first-response: 116.423 mean, 3318.18 max, 29.481 min
HTTP response codes:
code 200 — 100000

+++ 10MByte +++

$ http_load -verbose -parallel 100 -fetches 1000 ./10m.urls

* Varnish 1.1.2:

— 60 secs, 100 fetches started, 0 completed, 100 current
— 120 secs, 196 fetches started, 96 completed, 100 current
— 180 secs, 231 fetches started, 131 completed, 100 current
— 240 secs, 304 fetches started, 204 completed, 100 current
— 300 secs, 389 fetches started, 289 completed, 100 current
— 360 secs, 434 fetches started, 334 completed, 100 current
— 420 secs, 509 fetches started, 409 completed, 100 current
— 480 secs, 585 fetches started, 485 completed, 100 current
— 540 secs, 637 fetches started, 537 completed, 100 current
— 600 secs, 714 fetches started, 614 completed, 100 current
— 660 secs, 786 fetches started, 686 completed, 100 current
— 720 secs, 844 fetches started, 744 completed, 100 current
— 780 secs, 915 fetches started, 815 completed, 100 current
— 840 secs, 987 fetches started, 887 completed, 100 current
1000 fetches, 100 max parallel, 1.04858e+10 bytes, in 899.718 seconds
1.04858e+07 mean bytes/connection
1.11146 fetches/sec, 1.16545e+07 bytes/sec
msecs/connect: 129.428 mean, 3128.09 max, 0.291 min
msecs/first-response: 1003.12 mean, 9619.7 max, 120.662 min
HTTP response codes:
code 200 — 1000

* Squid 2.6.STABLE18:

— 60 secs, 103 fetches started, 3 completed, 100 current
— 120 secs, 178 fetches started, 78 completed, 100 current
— 180 secs, 255 fetches started, 155 completed, 100 current
— 240 secs, 322 fetches started, 222 completed, 100 current
— 300 secs, 379 fetches started, 279 completed, 100 current
— 360 secs, 458 fetches started, 358 completed, 100 current
— 420 secs, 518 fetches started, 418 completed, 100 current
— 480 secs, 583 fetches started, 483 completed, 100 current
— 540 secs, 661 fetches started, 561 completed, 100 current
— 600 secs, 721 fetches started, 621 completed, 100 current
— 660 secs, 786 fetches started, 686 completed, 100 current
— 720 secs, 863 fetches started, 763 completed, 100 current
— 780 secs, 926 fetches started, 826 completed, 100 current
— 840 secs, 984 fetches started, 884 completed, 100 current
1000 fetches, 100 max parallel, 1.04858e+10 bytes, in 894.062 seconds
1.04858e+07 mean bytes/connection
1.11849 fetches/sec, 1.17282e+07 bytes/sec
msecs/connect: 137.644 mean, 9128.16 max, 0.283 min
msecs/first-response: 131.811 mean, 3509.99 max, 28.891 min
HTTP response codes:
code 200 — 1000

* Squid 3.0.STABLE2:

— 60 secs, 105 fetches started, 5 completed, 100 current
— 120 secs, 190 fetches started, 90 completed, 100 current
http://benchmark.lo/10m/files-2/2: timed out
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-1/1: timed out
http://benchmark.lo/10m/files-1/1: byte count wrong
http://benchmark.lo/10m/files-5/2: timed out
http://benchmark.lo/10m/files-5/2: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-1/1: byte count wrong
http://benchmark.lo/10m/files-2/1: timed out
http://benchmark.lo/10m/files-2/1: byte count wrong
http://benchmark.lo/10m/files-1/1: byte count wrong
http://benchmark.lo/10m/files-2/1: byte count wrong
http://benchmark.lo/10m/files-2/1: byte count wrong
http://benchmark.lo/10m/files-4/2: timed out
http://benchmark.lo/10m/files-4/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-4/2: byte count wrong
http://benchmark.lo/10m/files-4/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-4/2: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-1/1: byte count wrong
http://benchmark.lo/10m/files-1/1: byte count wrong
http://benchmark.lo/10m/files-4/2: byte count wrong
http://benchmark.lo/10m/files-2/1: byte count wrong
http://benchmark.lo/10m/files-2/1: byte count wrong
http://benchmark.lo/10m/files-1/1: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-2/1: byte count wrong
http://benchmark.lo/10m/files-4/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-4/2: byte count wrong
— 180 secs, 260 fetches started, 160 completed, 100 current
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-1/1: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-2/1: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-3/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-2/1: byte count wrong
http://benchmark.lo/10m/files-4/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
http://benchmark.lo/10m/files-2/2: byte count wrong
— 240 secs, 315 fetches started, 215 completed, 100 current
— 300 secs, 395 fetches started, 295 completed, 100 current
— 360 secs, 464 fetches started, 364 completed, 100 current
— 420 secs, 521 fetches started, 421 completed, 100 current
— 480 secs, 596 fetches started, 496 completed, 100 current
— 540 secs, 664 fetches started, 564 completed, 100 current
— 600 secs, 725 fetches started, 625 completed, 100 current
— 660 secs, 798 fetches started, 698 completed, 100 current
— 720 secs, 865 fetches started, 765 completed, 100 current
— 780 secs, 929 fetches started, 829 completed, 100 current
— 840 secs, 997 fetches started, 897 completed, 100 current
1000 fetches, 100 max parallel, 1.03922e+10 bytes, in 886.272 seconds
1.03922e+07 mean bytes/connection
1.12832 fetches/sec, 1.17258e+07 bytes/sec
msecs/connect: 130.639 mean, 3127.72 max, 0.21 min
msecs/first-response: 186.15 mean, 3578.41 max, 34.601 min
5 timeouts
53 bad byte counts
HTTP response codes:
code 200 — 1000

proxy server status

+++ 100KByte +++

100k.varnish.iostat_xm5.log
100k.varnish.vmstat5.log
100k.squid2.iostat_xm5.log
100k.squid2.vmstat5.log
100k.squid3.iostat_xm5.log
100k.squid3.vmstat5.log

+++ 10MByte +++

10m.varnish.iostat_xm5.log
10m.varnish.vmstat5.log
10m.squid2.iostat_xm5.log
10m.squid2.vmstat5.log
10m.squid3.iostat_xm5.log
10m.squid3.vmstat5.log

After finished each benchmark, the status is as follow:

conclusion

The data here, the reference in your mind.
That which is the better choice as a caching proxy, depends on your own judgement.

notes

1. Varnish can cache files immediately.
I found this while comparing the fetching of 10MB files.
Varnish just request the nginx server for the file only once, but Squid will requests them for about
10 times of varnish does.

2. Varnish provides easier access to proxy status.

3. Squid is more than a web proxy, while Varnish is more than an http accelerator (cache proxy).
The Varnish web site claims that Varnish is ten to twenty times faster than the popular Squid
cache on the same hardware.

[...]
simple-vhost.server-root = "/var/web"
simple-vhost.default-host = "test.lo"
simple-vhost.document-root = "/www"
$HTTP["host"] =~ "^(www\.)?test\.lo$" {
        server.document-root = "/var/web/www"
}
$HTTP["host"] =~ "^blog\.test\.lo$" {
        server.document-root = "/var/web/blog"
}
[...]

Varnish on server 192.168.123.249 VCL file:

backend test {
set backend.host = "192.168.123.248";
set backend.port = "80";
}

acl purge {
"localhost";
"127.0.0.1";
}

sub vcl_recv {
if (req.http.host ~ "^(www|blog\.)?test\.lo$") {
set req.backend = test;
}

else {
error 404 "Unknown virtual host";
}
if (req.request == "PURGE") {
if (!client.ip ~ purge) {
error 405 "Not allowed.";
}
}
if (req.http.Cache-Control ~ "no-cache") {
pass;
}
if (req.request != "GET" && req.request != "HEAD") {
pipe;
}
if (req.http.Expect) {
pipe;
}
if (req.http.Authenticate || req.http.Cookie) {
pass;
}
lookup;
}

sub vcl_pipe {
pipe;
}

sub vcl_pass {
pass;
}

sub vcl_hash {
set req.hash += req.url;
set req.hash += req.http.host;
set req.hash += req.http.cookie;
hash;
}

sub vcl_hit {
if (req.request == "PURGE") {
set obj.ttl = 0s;
error 200 "Purged.";
}
if (!obj.cacheable) {
pass;
}
deliver;
}

sub vcl_miss {
if (req.request == "PURGE") {
error 404 "Not in cache.";
}
fetch;
}

sub vcl_fetch {
if (!obj.valid) {
error;
}
if (!obj.cacheable || obj.http.Set-Cookie|| \
        obj.http.Pragma ~ "no-cache" || \
        obj.http.Cache-Control ~ "no-cache" || \
        obj.http.Cache-Control ~ "private") {
pass;
}
if (req.request == "GET" && req.url ~ "\.(txt|js)$") {
set obj.ttl = 3600s;
}
else {
set obj.ttl = 30d;
}
insert;
}

sub vcl_deliver {
deliver;
}

sub vcl_timeout {
discard;
}

sub vcl_discard {
discard;
}

Above is just a simple example, it works for me currently.
Many people want to see the performance comparison between varnish and squid. Sorry but for now I couldn’t give that, maybe one day;)
I’m afraid that there’s going to be a variety of troubles to shooting, yeah, definitely.

• 1 History
• 2 Architecture
• 3 Performance
• 4 External links

History

The project was initiated by the online branch of the Norwegian tabloid newspaper

Architecture

Varnish is heavily

Performance

While Varnish is designed to reduce contention between threads to a minimum, its performance will only be as good as that of the system’s

External links

• official web site