About two weeks ago, I did a simple benchmark on varnish and squid, and wrote this Benchmark Caching of Varnish and Squid post.
Today Willy Tarreau gave me some advise. So I took a short time re-ran this benchmark. Here follows the detail.
platform
Everything kept unchanged except for the proxy_server, I replaced it with this new one:
o OS: Linux 2.6.21.5-smp i686 (Slackware 12.0)
o CPU: Intel(R) Xeon(TM) CPU 3.06GHz x 2
o MEM: 1024M x 6
o DISK: RAID 5
o Ethernet controller: Intel Corporation 82546EB Gigabit Ethernet Controller
file pool
This time, I generated 2 sets of 10Mbyte files, one is 1,000 files of 10kbyte size, and the other is 10 files of 1MByte size.
benchmark
* client: http_load
* proxy server: varnish 1.1.2, squid 2.6.STABLE18, and squid 3.0.STABLE2.
* http server: nginx/0.6.28
Using the same configurations of nginx, varnish, and squid 2/3, got the results below:
+++ 10KByte +++
$ http_load -verbose -parallel 100 -fetches 100000 ./10k.urls
100000 fetches, 100 max parallel, 1.024e+09 bytes, in 14.7505 seconds
10240 mean bytes/connection
6779.43 fetches/sec, 6.94213e+07 bytes/sec
msecs/connect: 0.400918 mean, 11.452 max, 0.067 min
msecs/first-response: 14.0161 mean, 1779.32 max, 0.24 min
HTTP response codes:
code 200 — 100000
* Squid 2.6.STABLE18:
100000 fetches, 100 max parallel, 1.024e+09 bytes, in 26.1771 seconds
10240 mean bytes/connection
3820.13 fetches/sec, 3.91181e+07 bytes/sec
msecs/connect: 0.497665 mean, 2990.79 max, 0.055 min
msecs/first-response: 21.0663 mean, 3018.84 max, 4.071 min
HTTP response codes:
code 200 — 100000
* Squid 3.0.STABLE2:
— 60.0027 secs, 100000 fetches started, 96249 completed, 0 current
100000 fetches, 100 max parallel, 9.85651e+08 bytes, in 102.375 seconds
9856.51 mean bytes/connection
976.8 fetches/sec, 9.62785e+06 bytes/sec
msecs/connect: 2.56114 mean, 91.428 max, 0.061 min
msecs/first-response: 27.8048 mean, 94.563 max, 1.288 min
3751 timeouts
3745 bad byte counts
HTTP response codes:
code 200 — 96255
+++ 1MByte +++
$ http_load -verbose -parallel 100 -fetches 1000 ./1m.urls
* Varnish 1.1.2:
— 60 secs, 6640 fetches started, 6540 completed, 100 current
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 91.1719 seconds
1.04858e+06 mean bytes/connection
109.683 fetches/sec, 1.15011e+08 bytes/sec
msecs/connect: 36.9187 mean, 9019.28 max, 0.08 min
msecs/first-response: 26.7986 mean, 475.462 max, 18.781 min
HTTP response codes:
code 200 — 10000
* Squid 2.6.STABLE18:
— 60 secs, 5856 fetches started, 5756 completed, 100 current
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 103.829 seconds
1.04858e+06 mean bytes/connection
96.3126 fetches/sec, 1.00991e+08 bytes/sec
msecs/connect: 2.4862 mean, 2994.65 max, 0.063 min
msecs/first-response: 15.9743 mean, 134.817 max, 8.222 min
HTTP response codes:
code 200 — 10000
* Squid 3.0.STABLE2:
— 60 secs, 6083 fetches started, 5983 completed, 100 current
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 103.054 seconds
1.04858e+06 mean bytes/connection
97.0367 fetches/sec, 1.0175e+08 bytes/sec
msecs/connect: 6.6513 mean, 3022.23 max, 0.089 min
msecs/first-response: 16.0642 mean, 787.308 max, 0.741 min
HTTP response codes:
code 200 — 10000
proxy server system status
+++ 10KByte +++
10k-varnish-iostat_xm5
10k-squid2-iostat_xm5
10k-squid3-iostat_xm5
10k-varnish-vmstat5
10k-squid2-vmstat5
10k-squid3-vmstat5
+++ 1MByte +++
1m-varnish-iostat_xm5
1m-squid2-iostat_xm5
1m-squid3-iostat_xm5
1m-varnish-vmstat5
1m-squid2-vmstat5
1m-squid3-vmstat5
proxy status
)-: Hmm, I’m still not satisfied with these results, especially the something about squid3.
Maybe I need to optimize the configuration of the proxies?
Comments 3
Nice update! This benchmark is quite better now. As you see in vmstat, with varnish, only system CPU is used and there is a lot of idle time remaining. That means that varnish is optimized to induce a very low overhead. On the other side, you see both squid versions. They use half user and half system time, and the CPU is saturated (50% idle on a dual proc means 1 proc saturated). So squid is limited here by its architecture and design. If it consumed less user time, this time would be better spent in system which would simply double the
performance, leading to approximately what you see in varnish. Maybe there are things that can be done with squid to optimize it, honnestly I don’t know. 25% user seems a lot to me on such a component, but since it’s a very flexible proxy I don’t know if there is an entry cost even if everything is disabled.
I think I will soon have to take a look at varnish, it looks promising :-)
Willy
Posted 05 Apr 2008 at 03:46 ¶Yeah, this time the result is much better :-)
Posted 05 Apr 2008 at 09:37 ¶We now have one project run varnish, I think it’s a better choice.
IMHO, there’s no doubt that varnish is more powerful by reading its architecture notes.
Thanks again for your time, Willy.
z558×9wco8g4crjt
Posted 13 Nov 2008 at 10:07 ¶Post a Comment