About two weeks ago, I did a simple benchmark on varnish and squid, and wrote this Benchmark Caching of Varnish and Squid post.
Today Willy Tarreau gave me some advise. So I took a short time re-ran this benchmark. Here follows the detail.
platform
Everything kept unchanged except for the proxy_server, I replaced it with this new one:
o OS: Linux 2.6.21.5-smp i686 (Slackware 12.0)
o CPU: Intel(R) Xeon(TM) CPU 3.06GHz x 2
o MEM: 1024M x 6
o DISK: RAID 5
o Ethernet controller: Intel Corporation 82546EB Gigabit Ethernet Controller
file pool
This time, I generated 2 sets of 10Mbyte files, one is 1,000 files of 10kbyte size, and the other is 10 files of 1MByte size.
benchmark
* client: http_load
* proxy server: varnish 1.1.2, squid 2.6.STABLE18, and squid 3.0.STABLE2.
* http server: nginx/0.6.28
Using the same configurations of nginx, varnish, and squid 2/3, got the results below:
+++ 10KByte +++
$ http_load -verbose -parallel 100 -fetches 100000 ./10k.urls
100000 fetches, 100 max parallel, 1.024e+09 bytes, in 14.7505 seconds
10240 mean bytes/connection
6779.43 fetches/sec, 6.94213e+07 bytes/sec
msecs/connect: 0.400918 mean, 11.452 max, 0.067 min
msecs/first-response: 14.0161 mean, 1779.32 max, 0.24 min
HTTP response codes:
code 200 — 100000
* Squid 2.6.STABLE18:
100000 fetches, 100 max parallel, 1.024e+09 bytes, in 26.1771 seconds
10240 mean bytes/connection
3820.13 fetches/sec, 3.91181e+07 bytes/sec
msecs/connect: 0.497665 mean, 2990.79 max, 0.055 min
msecs/first-response: 21.0663 mean, 3018.84 max, 4.071 min
HTTP response codes:
code 200 — 100000
* Squid 3.0.STABLE2:
— 60.0027 secs, 100000 fetches started, 96249 completed, 0 current
100000 fetches, 100 max parallel, 9.85651e+08 bytes, in 102.375 seconds
9856.51 mean bytes/connection
976.8 fetches/sec, 9.62785e+06 bytes/sec
msecs/connect: 2.56114 mean, 91.428 max, 0.061 min
msecs/first-response: 27.8048 mean, 94.563 max, 1.288 min
3751 timeouts
3745 bad byte counts
HTTP response codes:
code 200 — 96255
+++ 1MByte +++
$ http_load -verbose -parallel 100 -fetches 1000 ./1m.urls
* Varnish 1.1.2:
— 60 secs, 6640 fetches started, 6540 completed, 100 current
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 91.1719 seconds
1.04858e+06 mean bytes/connection
109.683 fetches/sec, 1.15011e+08 bytes/sec
msecs/connect: 36.9187 mean, 9019.28 max, 0.08 min
msecs/first-response: 26.7986 mean, 475.462 max, 18.781 min
HTTP response codes:
code 200 — 10000
* Squid 2.6.STABLE18:
— 60 secs, 5856 fetches started, 5756 completed, 100 current
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 103.829 seconds
1.04858e+06 mean bytes/connection
96.3126 fetches/sec, 1.00991e+08 bytes/sec
msecs/connect: 2.4862 mean, 2994.65 max, 0.063 min
msecs/first-response: 15.9743 mean, 134.817 max, 8.222 min
HTTP response codes:
code 200 — 10000
* Squid 3.0.STABLE2:
— 60 secs, 6083 fetches started, 5983 completed, 100 current
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 103.054 seconds
1.04858e+06 mean bytes/connection
97.0367 fetches/sec, 1.0175e+08 bytes/sec
msecs/connect: 6.6513 mean, 3022.23 max, 0.089 min
msecs/first-response: 16.0642 mean, 787.308 max, 0.741 min
HTTP response codes:
code 200 — 10000
proxy server system status
+++ 10KByte +++
10k-varnish-iostat_xm5
10k-squid2-iostat_xm5
10k-squid3-iostat_xm5
10k-varnish-vmstat5
10k-squid2-vmstat5
10k-squid3-vmstat5
+++ 1MByte +++
1m-varnish-iostat_xm5
1m-squid2-iostat_xm5
1m-squid3-iostat_xm5
1m-varnish-vmstat5
1m-squid2-vmstat5
1m-squid3-vmstat5
proxy status
)-: Hmm, I’m still not satisfied with these results, especially the something about squid3.
Maybe I need to optimize the configuration of the proxies?
Nice update! This benchmark is quite better now. As you see in vmstat, with varnish, only system CPU is used and there is a lot of idle time remaining. That means that varnish is optimized to induce a very low overhead. On the other side, you see both squid versions. They use half user and half system time, and the CPU is saturated (50% idle on a dual proc means 1 proc saturated). So squid is limited here by its architecture and design. If it consumed less user time, this time would be better spent in system which would simply double the
performance, leading to approximately what you see in varnish. Maybe there are things that can be done with squid to optimize it, honnestly I don’t know. 25% user seems a lot to me on such a component, but since it’s a very flexible proxy I don’t know if there is an entry cost even if everything is disabled.
I think I will soon have to take a look at varnish, it looks promising :-)
Willy
Hi Willy,
Yeah, this time the result is much better :-)
We now have one project run varnish, I think it’s a better choice.
IMHO, there’s no doubt that varnish is more powerful by reading its architecture notes.
Thanks again for your time, Willy.
Could you post your configs for varnish and squid? That would be useful to know. Thanks.
Hi Jauder,
As this benchmark was taken one year ago, I cannot find the configurations. Sorry :-(
Just a word of caution: the current version of WordPress is 2.8.2 and it seems you still have 2.7.something. There was a significant security fix in 2.8.1 so you may want to upgrade sooner rather than later. Here is a quick upgrade quide that may be helpful: http://www.krazyworks.com/?p=812
great article, will recommend to friends rep
@Demaray I’m not using 2.7 at that time. Anyway, thanks for your advice, and your great article.
@cowboydeals Thank you guy!
Good stuff – I have also done some benchmarking of my own on Squid and Varnish. My results found that Varnish serves up content much faster than Squid does. My tests really test the throughput of the two applications.
http://deserialized.com/reverse-proxy-performance-varnish-vs-squid-part-1/
@Bryan Beautiful test, waiting for the part2. Thanks for share:-)