<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	>

<channel>
	<title>IScale &#187; linux</title>
	<atom:link href="http://dotimes.com/iscale/category/linux/feed" rel="self" type="application/rss+xml" />
	<link>http://dotimes.com/iscale</link>
	<description>Living within Dot Times</description>
	<pubDate>Tue, 18 Nov 2008 06:04:39 +0000</pubDate>
	<generator>http://wordpress.org/?v=2.6.3</generator>
	<language>en</language>
			<item>
		<title>Building Customized Installation CD/DVD of Slackware</title>
		<link>http://dotimes.com/iscale/2008/04/building-customized-installation-cddvd-of-slackware.html</link>
		<comments>http://dotimes.com/iscale/2008/04/building-customized-installation-cddvd-of-slackware.html#comments</comments>
		<pubDate>Sat, 12 Apr 2008 09:05:32 +0000</pubDate>
		<dc:creator>Cherife Li</dc:creator>
		
		<category><![CDATA[linux]]></category>

		<category><![CDATA[slackware]]></category>

		<guid isPermaLink="false">http://dotimes.com/iscale/?p=81</guid>
		<description><![CDATA[In order to bring off one-command or even auto installation, and save time of the installation process, the idea that building an Slackware installation CD/DVD came to my mind.
What to do and how to do, this was a question to me at the beginning. I have no idea about isolinux. Fortunately I found that there [...]]]></description>
			<content:encoded><![CDATA[<p>In order to bring off one-command or even auto installation, and save time of the installation process, the idea that building an Slackware installation CD/DVD came to my mind.</p>
<p>What to do and how to do, this was a question to me at the beginning. I have no idea about isolinux. Fortunately I found that there is an answer called <a href="http://www.linuxquestions.org/linux/answers/Applications_GUI_Multimedia/Cooking_Up_Some_Slack_CUSS" target="_blank">Cooking Up Some Slack(CUSS)</a>. After reading over that, I got to know something and began the process.</p>
<p><strong>1 Prepare the packages</strong><br />
I pulled the -Current repository from rsync://rsync.osuosl.org/slackware/slackware-current to my workstation. Some directories and files are not necessary for the installation, to save bandwidth and time, it is sensible to &#8211;exclude them, like source/, pasture/, testing/, and so on.</p>
<p><strong>2 Customize initrd.img</strong><br />
The initrd.img contains the system and scripts for installation. Most things will be customized here.</p>
<p>First, uncompress the initrd.img file to a directory. The initrd.img in -Current now is a gzipped cpio archive, rather than a gzipped image as in former versions(before and include 11.0).</p>
<p><code>$ gzip -cd initrd.img|cpio -i</code></p>
<p>Most of the installation scripts are stored in usr/lib/setup directory.</p>
<p>Second, to reduce the size of initrd.img, I deleted some directories, e.g., lib/modules/&#8217;kernel-version&#8217;(retain the -smp directory, I just use -smp to boot), usr/lib/setup(you need installpkg script at least), usr/man, and some files, e.g., scripts related to LVM in sbin/ directory(I don&#8217;t use LVM), odd files in etc/ directory, and so on.</p>
<p>Third, customize the environment and script during installation. The files related are etc/profile, etc/rc.d/rc.S, and also etc/inittab, etc/issue and so on, if you want. </p>
<p>This is some pieces of my setup script:</p>
<blockquote><p>
#!/bin/sh<br />
#<br />
# Copyright (c) 2008, Cherife Li <cherife@dotimes.com>.<br />
# All rights reserved.<br />
#<br />
# Redistribution and use of this script, with or without modification,<br />
# is permitted, and must retain the copyright above.<br />
#<br />
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR &#8220;AS IS&#8221; AND ANY EXPRESS OR<br />
# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED<br />
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE<br />
# DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT,<br />
# INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES<br />
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR<br />
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)<br />
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,<br />
# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING<br />
# IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE<br />
# POSSIBILITY OF SUCH DAMAGE.<br />
#<br />
# Notice:<br />
# - This script is used for one logical disk(not LVM).<br />
# - The disk will be formatted to reiserfs file system.<br />
# - I don&#8217;t care about the keyboard map setting.<br />
#<br />
# Bugs, advise, suggestion, or others, if any, please mail me.<br />
#</p>
<p>TMP=/var/log/setup/tmp<br />
SRC=/var/log/setup/src<br />
DEST=/mnt</p>
<p>KERNEL=&#8221;`cut -f 5 -d &#8216; &#8216; /proc/cmdline|cut -f 2 -d =`&#8221;<br />
KNAME=&#8221;huge-smp&#8221;<br />
KVERSION=&#8221;`uname -r`&#8221;</p>
<p>if [ -d $TMP ]; then<br />
  rm -rf $TMP/* 1>/dev/null 2>&#038;1<br />
else<br />
  mkdir $TMP<br />
fi</p>
<p>for i in $SRC $DEST; do<br />
  if [ -d $i ]; then<br />
    umount -f $i 1>/dev/null 2>&#038;1<br />
  else<br />
    mkdir -p $i 1>/dev/null 2>&#038;1<br />
  fi<br />
done</p>
<p>echo<br />
echo &#8220;+==============================================+&#8221;<br />
echo &#8221; Preparing the hard disk for installation &#8230;&#8221;<br />
echo<br />
sleep 3</p>
<p>DISK=`fdisk -l|grep &#8216;^Disk\ \/&#8217;|cut -f 2 -d &#8216; &#8216;|cut -f 1 -d :|head -n1`</p>
<p># clear existing partition(s), if any<br />
PARTNUM=&#8221;`fdisk $DISK -l|tail -n1|cut -f 1 -d &#8216; &#8216;|cut -c 9`&#8221;<br />
while [ $PARTNUM -gt 0 ]; do<br />
  if [ $PARTNUM -gt 1 ]; then<br />
    echo -ne &#8220;d\n$PARTNUM\n&#8221; >> $TMP/fdisk.del<br />
  else<br />
    echo -ne &#8220;d\nw\n&#8221; >> $TMP/fdisk.del<br />
  fi<br />
  let PARTNUM=PARTNUM-1<br />
done<br />
cat $TMP/fdisk.del|fdisk $DISK</p>
<p># Partition the hard disk into two partitions.<br />
# one for swap with 2000M, the other for root(/) with the left of disk.<br />
echo -ne &#8220;n\np\n1\n\n+2000M\n\nn\np\n2\n\n\nt\n1\n82\nw\n&#8221;|fdisk $DISK</p>
<p># Active swap partition and add it to fstab temp<br />
SWAP_PART=${DISK}1<br />
mkswap -v1 $SWAP_PART<br />
swapon $SWAP_PART<br />
printf &#8220;%-16s %-16s %-11s %-16s %-3s %s\n&#8221; &#8220;$SWAP_PART&#8221; &#8220;swap&#8221; &#8220;swap&#8221; &#8220;defaults&#8221; &#8220;0&#8243; &#8220;0&#8243; >> $TMP/fstab.tmp</p>
<p># Format root (/) partition, mount it, and add it to fstab temp<br />
ROOT_PART=${DISK}2<br />
echo &#8220;y&#8221;|mkreiserfs $ROOT_PART 1>/dev/null 2>&#038;1<br />
mount $ROOT_PART $DEST || exit 1<br />
printf &#8220;%-16s %-16s %-11s %-16s %-3s %s\n&#8221; &#8220;$ROOT_PART&#8221; &#8220;/&#8221; &#8220;reiserfs&#8221; &#8220;defaults,noatime&#8221; &#8220;1&#8243; &#8220;1&#8243; >> $TMP/fstab.tmp</p>
<p>echo<br />
echo &#8221; The hard disk is ready for installation.&#8221;<br />
echo &#8220;+=============================================+&#8221;<br />
echo<br />
#sleep 3</p>
<p>echo<br />
echo &#8220;+=============================================+&#8221;<br />
echo &#8221; Processing the package installation &#8230;&#8221;<br />
echo<br />
sleep 3</p>
<p># Mount the Slackware source<br />
for MEDIA in \<br />
  /dev/hdc /dev/hdd /dev/hda /dev/hdb \<br />
  /dev/hde /dev/hdf /dev/hdg /dev/hdh \<br />
  /dev/hdi /dev/hdj /dev/hdk /dev/hdl \<br />
  /dev/sr0 /dev/sr1 /dev/sr2 /dev/sr3 \<br />
  /dev/pcd0 /dev/pcd1 /dev/pcd2 /dev/pcd3;<br />
do<br />
  mount -t iso9660 -o ro $MEDIA $SRC 1>/dev/null 2>&#038;1<br />
  if [ $? = 0 ]; then<br />
    break<br />
  fi<br />
done</p>
<p>if [ "`mount|grep "$SRC"`" = "" ]; then<br />
  echo<br />
  echo &#8220;Couldn&#8217;t found the installation source!&#8221;<br />
  echo &#8220;Abort &#8230;&#8221;<br />
  echo<br />
  exit 1<br />
else<br />
  # Process the installation<br />
  echo<br />
  echo &#8220;Found the installation source.&#8221;<br />
  echo &#8220;Now going to install the packages &#8230;&#8221;<br />
  echo<br />
  sleep 3<br />
  for series in a ap d l n; # you may do some change here<br />
  do<br />
    installpkg -root $DEST -priority ADD $SRC/slackware/$series/*.tgz<br />
  done</p>
<p>  echo<br />
  echo &#8221; All packages have been installed.&#8221;<br />
  echo &#8220;+=============================================+&#8221;<br />
  echo<br />
  sleep 3<br />
fi</p>
<p># config system<br />
echo<br />
echo &#8220;+==============================================+&#8221;<br />
echo &#8221; Configuring the system, wait for a while &#8230;&#8221;<br />
echo</p>
<p># Permission issue<br />
chmod 755 $DEST $DEST/var/spool/mail 1>dev/null 2>&#038;1</p>
<p># Kernel setting<br />
(cd $DEST/boot<br />
rm -rf vmlinuz config System.map 1>/dev/null 2>&#038;1<br />
cp -a $SRC/kernels/$KERNEL/System.map.gz ./System.map-$KNAME-$KVERSION.gz<br />
cp -a $SRC/kernels/$KERNEL/bzImage ./vmlinuz-$KNAME-$KVERSION<br />
cp -a $SRC/kernels/$KERNEL/config ./config-$KNAME-$KVERSION<br />
gzip -df System.map-$KNAME-$KVERSION.gz<br />
ln -sf vmlinuz-$KNAME-$KVERSION vmlinuz<br />
ln -sf config-$KNAME-$KVERSION config<br />
ln -sf System.map-$KNAME-$KVERSION System.map<br />
)</p>
<p># Boot loader<br />
cat > $DEST/etc/lilo.conf << EOF<br />
boot = $DISK<br />
timeout = 0<br />
change-rules<br />
reset<br />
vga = normal<br />
image = /boot/vmlinuz<br />
root = $ROOT_PART<br />
label = default<br />
read-only<br />
EOF<br />
lilo -r $DEST -C /etc/lilo.conf</p>
<p># Network<br />
for nicm in \<br />
  e1000 tg3 eepro100 epic100 8139too 8139cp pcnet32 hp100 ne2k-pci olympic rcpci 3c59x acenic de4x5 dgrs sktr skge tulip via-rhine yellowfin dl2k ns83820;<br />
do<br />
  modprobe $nicm 2> /dev/null<br />
  grep -q eth /proc/net/dev<br />
  if [ $? = 0 ]; then<br />
    echo &#8220;/sbin/modprobe $nicm&#8221; > $DEST/etc/rc.d/rc.netdevice<br />
    chmod 755 $DEST/etc/rc.d/rc.netdevice<br />
    break<br />
  else<br />
    rmmod $nicm 2> /dev/null<br />
  fi<br />
done</p>
<p>if [ ! -f $DEST/etc/rc.d/rc.netdevice ]; then<br />
  echo<br />
  echo &#8220;Warning: No network interface card found.&#8221;<br />
  echo &#8220;Maybe, you need to check the card(s) or the modules.&#8221;<br />
  echo<br />
fi </p>
<p>[... Network setups, e.g., $DEST/etc/rc.d/rc.inet1.conf, $DEST/etc/networks, \<br />
     $DEST/etc/hosts, $DEST/etc/HOSTNAME, $DEST/etc/resolv.conf, here \<br />
  snip...]</p>
<p># Time config<br />
cat $DEST/usr/share/zoneinfo/Asia/Shanghai > $DEST/etc/localtime<br />
cat > $DEST/etc/hardwareclocl << EOF<br />
localtime<br />
EOF</p>
<p># The /etc/fstab file<br />
cat $TMP/fstab.tmp > $DEST/etc/fstab<br />
printf &#8220;%-16s %-16s %-11s %-16s %-3s %s\n&#8221; &#8220;#/dev/cdrom&#8221; &#8220;/mnt/cdrom&#8221; &#8220;auto&#8221; &#8220;noauto,owner,ro&#8221; &#8220;0&#8243; &#8220;0&#8243; >> $DEST/etc/fstab<br />
printf &#8220;%-16s %-16s %-11s %-16s %-3s %s\n&#8221; &#8220;#/dev/fd0&#8243; &#8220;/mnt/floppy&#8221; &#8220;auto&#8221; &#8220;noauto,owner&#8221; &#8220;0&#8243; &#8220;0&#8243; >> $DEST/etc/fstab<br />
printf &#8220;%-16s %-16s %-11s %-16s %-3s %s\n&#8221; &#8220;devpts&#8221; &#8220;/dev/pts&#8221; &#8220;devpts&#8221; &#8220;gid=5,mode=620&#8243; &#8220;0&#8243; &#8220;0&#8243; >> $DEST/etc/fstab<br />
printf &#8220;%-16s %-16s %-11s %-16s %-3s %s\n&#8221; &#8220;proc&#8221; &#8220;/proc&#8221; &#8220;proc&#8221; &#8220;defaults&#8221; &#8220;0&#8243; &#8220;0&#8243; >> $DEST/etc/fstab<br />
printf &#8220;%-16s %-16s %-11s %-16s %-3s %s\n&#8221; &#8220;tmpfs&#8221; &#8220;/dev/shm&#8221; &#8220;tmpfs&#8221; &#8220;defaults&#8221; &#8220;0&#8243; &#8220;0&#8243; >> $DEST/etc/fstab</p>
<p># Misc<br />
cat > $DEST/etc/sysctl.conf << EOF<br />
[... snip ...]<br />
EOF</p>
<p>rm -f $DEST/var/spool/mail/root<br />
sed -i '/^c[2-6].*agetty/s/^/#/' $DEST/etc/inittab<br />
sed -i '/^\/sbin\/modprobe.*[parport_pc|lp|agpgart]/s/^/#/' $DEST/etc/rc.d/rc.modules<br />
echo "blacklist ipv6" >> $DEST/etc/modprobe.d/blacklist<br />
chroot $DEST /sbin/ldconfig<br />
chroot $DEST /usr/bin/passwd root</p>
<p># All finished<br />
echo<br />
echo &#8220;+=========================================================+&#8221;<br />
echo &#8221; Wooow, all done.&#8221;<br />
echo &#8221; Now you may reboot with CTRL-ALT-DELETE, enjoy. ;-)&#8221;<br />
echo &#8220;+=========================================================+&#8221;<br />
echo<br />
cat /etc/wel.come
</p></blockquote>
<p>Last, rebuild the initrd.img.</p>
<p><code>$ find . | cpio -o -H newc | gzip -9 > /PATH/TO/isolinux/initrd.img</code></p>
<p><strong>3 Customize the kernel</strong><br />
Re-build the kernel with your own config, then copy System.map.gz, bzImage, config, to the kernels/huge-smp directory, or an new directory in kernels/. I removed the &#8216;kernel-version&#8217; subdirectory of kernels/, as I just use the -smp kernel.<br />
Then modify isolinux/isolinux.cfg to fit the changes.</p>
<p><strong>4 Specify the packages to install</strong><br />
In the slackware/ directory, I just put the packages needed here, and removed the unneeded packages. (The setup script will install all the packages in the series directories.) It is possible to use tagfile to install packages. But I just want a small ISO to build the installation CD/DVD. Which method to use depends on the need. The former is preferred to my situation. If using the later, slackbasics.org has <a href="http://www.slackbasics.org/html/chap-pkgmgmt.html#chap-pkgmgmt-tagfiles" target="_blank">an intro to the tagfiles</a>.<br />
I also added some packages built by myself. :-)</p>
<p><strong>5 Build the ISO image</strong><br />
At the top directory of the ISO source,<br />
<code>$ mkisofs -o /tmp/slackware-install-cd.iso -R -J -V "Slackware Install CD" -hide-rr-moved -v -d -N -no-emul-boot -boot-load-size 4 -boot-info-table -sort isolinux/iso.sort -b isolinux/isolinux.bin -c isolinux/isolinux.boot -A Slack-CD .</code>(do not forget this dot)<br />
then this ISO image could be used to burn an install CD/DVD.</p>
<p>That&#8217;s all.</p>
<p>Notes:</p>
<li>I&#8217;d prefer to make an auto install CD/DVD(just put the setup script path in etc/rc.d/rc.S), and auto reboot when the installation finished(put an reboot command at the end of the setup script). I would just need to boot the machine with CD/DVD then left for WC or smoking, when back, the system was up and ready for use. lol</li>
<li>It&#8217;s suitable to make an install CD/DVD for other versions of Slackware.</li>
<li>It&#8217;s also suitable to make an image for network(PXE) booting and [auto ]installation. I think it&#8217;s more sensible and necessary.</li>
]]></content:encoded>
			<wfw:commentRss>http://dotimes.com/iscale/2008/04/building-customized-installation-cddvd-of-slackware.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>Benchmark Caching of Varnish and Squid Again</title>
		<link>http://dotimes.com/iscale/2008/04/benchmark-caching-of-varnish-and-squid-again.html</link>
		<comments>http://dotimes.com/iscale/2008/04/benchmark-caching-of-varnish-and-squid-again.html#comments</comments>
		<pubDate>Tue, 01 Apr 2008 08:06:35 +0000</pubDate>
		<dc:creator>Cherife Li</dc:creator>
		
		<category><![CDATA[linux]]></category>

		<category><![CDATA[proxy]]></category>

		<category><![CDATA[nginx]]></category>

		<category><![CDATA[squid]]></category>

		<category><![CDATA[varnish]]></category>

		<guid isPermaLink="false">http://dotimes.com/iscale/?p=40</guid>
		<description><![CDATA[About two weeks ago, I did a simple benchmark on varnish and squid, and wrote this Benchmark Caching of Varnish and Squid post.
Today Willy Tarreau gave me some advise. So I took a short time re-ran this benchmark. Here follows the detail.
platform
Everything kept unchanged except for the proxy_server, I replaced it with this new one:

 [...]]]></description>
			<content:encoded><![CDATA[<p>About two weeks ago, I did a simple benchmark on varnish and squid, and wrote this <a href="http://dotimes.com/iscale/2008/03/benchmark-caching-of-varnish-and-squid.html">Benchmark Caching of Varnish and Squid</a> post.</p>
<p>Today <a href="http://1wt.eu/">Willy Tarreau</a> gave me <a href="http://dotimes.com/iscale/2008/03/benchmark-caching-of-varnish-and-squid.html#comment-55">some advise</a>. So I took a short time re-ran this benchmark. Here follows the detail.</p>
<p><strong>platform</strong><br />
Everything kept unchanged except for the <em>proxy_server</em>, I replaced it with this new one:</p>
<pre>
    o OS: Linux 2.6.21.5-smp i686 (Slackware 12.0)
    o CPU:  Intel(R) Xeon(TM) CPU 3.06GHz x 2
    o MEM: 1024M x 6
    o DISK: RAID 5
    o Ethernet controller:  Intel Corporation 82546EB Gigabit Ethernet Controller
</pre>
<p><strong>file pool</strong><br />
This time, I generated 2 sets of 10Mbyte files, one is 1,000 files of 10kbyte size, and the other is 10 files of 1MByte size.</p>
<p><strong>benchmark</strong></p>
<pre>
    * client: http_load
    * proxy server: varnish 1.1.2, squid 2.6.STABLE18, and squid 3.0.STABLE2.
    * http server: nginx/0.6.28
</pre>
<p>Using the same configurations of nginx, varnish, and squid 2/3, got the results below:<br />
<br />
+++ 10KByte +++<br />
<br />
$ <code>http_load -verbose -parallel 100 -fetches 100000 ./10k.urls</code><br />
</p>
<blockquote><p>
100000 fetches, 100 max parallel, 1.024e+09 bytes, in 14.7505 seconds<br />
10240 mean bytes/connection<br />
6779.43 fetches/sec, 6.94213e+07 bytes/sec<br />
msecs/connect: 0.400918 mean, 11.452 max, 0.067 min<br />
msecs/first-response: 14.0161 mean, 1779.32 max, 0.24 min<br />
HTTP response codes:<br />
  code 200 &#8212; 100000
</p></blockquote>
<p>* Squid 2.6.STABLE18:</p>
<blockquote><p>
100000 fetches, 100 max parallel, 1.024e+09 bytes, in 26.1771 seconds<br />
10240 mean bytes/connection<br />
3820.13 fetches/sec, 3.91181e+07 bytes/sec<br />
msecs/connect: 0.497665 mean, 2990.79 max, 0.055 min<br />
msecs/first-response: 21.0663 mean, 3018.84 max, 4.071 min<br />
HTTP response codes:<br />
  code 200 &#8212; 100000
</p></blockquote>
<p>* Squid 3.0.STABLE2:</p>
<blockquote><p>
&#8212; 60.0027 secs, 100000 fetches started, 96249 completed, 0 current<br />
100000 fetches, 100 max parallel, 9.85651e+08 bytes, in 102.375 seconds<br />
9856.51 mean bytes/connection<br />
976.8 fetches/sec, 9.62785e+06 bytes/sec<br />
msecs/connect: 2.56114 mean, 91.428 max, 0.061 min<br />
msecs/first-response: 27.8048 mean, 94.563 max, 1.288 min<br />
3751 timeouts<br />
3745 bad byte counts<br />
HTTP response codes:<br />
  code 200 &#8212; 96255
</p></blockquote>
<p>+++ 1MByte +++<br />
<br />
$ <code>http_load -verbose -parallel 100 -fetches 1000 ./1m.urls</code><br />
<br />
* Varnish 1.1.2:</p>
<blockquote><p>
&#8212; 60 secs, 6640 fetches started, 6540 completed, 100 current<br />
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 91.1719 seconds<br />
1.04858e+06 mean bytes/connection<br />
109.683 fetches/sec, 1.15011e+08 bytes/sec<br />
msecs/connect: 36.9187 mean, 9019.28 max, 0.08 min<br />
msecs/first-response: 26.7986 mean, 475.462 max, 18.781 min<br />
HTTP response codes:<br />
  code 200 &#8212; 10000
</p></blockquote>
<p>* Squid 2.6.STABLE18:</p>
<blockquote><p>
&#8212; 60 secs, 5856 fetches started, 5756 completed, 100 current<br />
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 103.829 seconds<br />
1.04858e+06 mean bytes/connection<br />
96.3126 fetches/sec, 1.00991e+08 bytes/sec<br />
msecs/connect: 2.4862 mean, 2994.65 max, 0.063 min<br />
msecs/first-response: 15.9743 mean, 134.817 max, 8.222 min<br />
HTTP response codes:<br />
  code 200 &#8212; 10000
</p></blockquote>
<p>* Squid 3.0.STABLE2:</p>
<blockquote><p>
&#8212; 60 secs, 6083 fetches started, 5983 completed, 100 current<br />
10000 fetches, 100 max parallel, 1.04858e+10 bytes, in 103.054 seconds<br />
1.04858e+06 mean bytes/connection<br />
97.0367 fetches/sec, 1.0175e+08 bytes/sec<br />
msecs/connect: 6.6513 mean, 3022.23 max, 0.089 min<br />
msecs/first-response: 16.0642 mean, 787.308 max, 0.741 min<br />
HTTP response codes:<br />
  code 200 &#8212; 10000
</p></blockquote>
<p><strong>proxy server system status</strong></p>
<p>+++ 10KByte +++</p>
<p><a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-varnish-iostat_xm5.log'>10k-varnish-iostat_xm5</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-squid2-iostat_xm5.log'>10k-squid2-iostat_xm5</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-squid3-iostat_xm5.log'>10k-squid3-iostat_xm5</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-varnish-vmstat5.log'>10k-varnish-vmstat5</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-squid2-vmstat5.log'>10k-squid2-vmstat5</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-squid3-vmstat5.log'>10k-squid3-vmstat5</a></p>
<p>+++ 1MByte +++</p>
<p><a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-varnish-iostat_xm5.log'>1m-varnish-iostat_xm5</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-squid2-iostat_xm5.log'>1m-squid2-iostat_xm5</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-squid3-iostat_xm5.log'>1m-squid3-iostat_xm5</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-varnish-vmstat5.log'>1m-varnish-vmstat5</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-squid2-vmstat5.log'>1m-squid2-vmstat5</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-squid3-vmstat5.log'>1m-squid3-vmstat5</a></p>
<p><strong>proxy status</strong></p>
<p><a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-varnishstat.png'><img src="http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-varnishstat-150x150.png" alt="" title="10k-varnishstat" width="150" height="150" class="alignnone size-thumbnail wp-image-75" /></a> <a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-squid2.png'><img src="http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-squid2-150x150.png" alt="" title="10k-squid2" width="150" height="150" class="alignnone size-thumbnail wp-image-79" /></a> <a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-squid3.png'><img src="http://dotimes.com/iscale/wp-content/uploads/2008/04/10k-squid3-150x150.png" alt="" title="10k-squid3" width="150" height="150" class="alignnone size-thumbnail wp-image-80" /></a><br />
<br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-varnishstat.png'><img src="http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-varnishstat-150x150.png" alt="" title="1m-varnishstat" width="150" height="150" class="alignnone size-thumbnail wp-image-78" /></a> <a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-squid2.png'><img src="http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-squid2-150x150.png" alt="" title="1m-squid2" width="150" height="150" class="alignnone size-thumbnail wp-image-76" /></a> <a href='http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-squid3.png'><img src="http://dotimes.com/iscale/wp-content/uploads/2008/04/1m-squid3-150x150.png" alt="" title="1m-squid3" width="150" height="150" class="alignnone size-thumbnail wp-image-77" /></a><br />
<br />
)-: Hmm, I&#8217;m still not satisfied with these results, especially the something about squid3.<br />
Maybe I need to optimize the configuration of the proxies?</p>
]]></content:encoded>
			<wfw:commentRss>http://dotimes.com/iscale/2008/04/benchmark-caching-of-varnish-and-squid-again.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>Benchmark Caching of Varnish and Squid</title>
		<link>http://dotimes.com/iscale/2008/03/benchmark-caching-of-varnish-and-squid.html</link>
		<comments>http://dotimes.com/iscale/2008/03/benchmark-caching-of-varnish-and-squid.html#comments</comments>
		<pubDate>Mon, 17 Mar 2008 16:45:01 +0000</pubDate>
		<dc:creator>Cherife Li</dc:creator>
		
		<category><![CDATA[linux]]></category>

		<category><![CDATA[proxy]]></category>

		<category><![CDATA[nginx]]></category>

		<category><![CDATA[squid]]></category>

		<category><![CDATA[varnish]]></category>

		<guid isPermaLink="false">http://dotimes.com/iscale/2008/03/benchmark-caching-of-varnish-and-squid.html</guid>
		<description><![CDATA[#
#   Readme
#
# I re-ran this benchmark and got new results in another post, which are more reasonable.
# Thanks to Willy Tarreau for pointing out the inadequacy in this benchmark.
# It&#8217;s recommended that you skimming this post and then referring to this fresh one.
#
&#8212;&#8212;&#8211;
Today, I finished a benchmark to compare the caching performance and [...]]]></description>
			<content:encoded><![CDATA[<p>#<br />
#   Readme<br />
#<br />
# I re-ran this benchmark and got new results in another post, which are more reasonable.<br />
# Thanks to <a href="http://1wt.eu/" target="_blank">Willy Tarreau</a> for pointing out the inadequacy in this benchmark.<br />
# It&#8217;s recommended that you skimming this post and then referring to <a href="http://dotimes.com/iscale/2008/04/benchmark-caching-of-varnish-and-squid-again.html" target="_blank">this fresh one</a>.<br />
#<br />
&#8212;&#8212;&#8211;</p>
<p>Today, I finished a benchmark to compare the caching performance and status between Varnish and Squid, which get widely focused on as reverse proxies.<br />
Here we go.</p>
<p><strong>platform</strong></p>
<p>The test-network is made up of:</p>
<pre>
    * D-Link 1024R, a 24-port Gigabit Switch
    * http_server:
          o OS: Linux 2.6.21.5-smp i686 (Slackware 12.0)
          o CPU: Intel(R) Xeon(TM) CPU 2.80GHz x 2
          o MEM: 1024M x 6
          o DISK: SEAGATE ST373405LC SCSI Disk
          o Ethernet controller: Intel 82546EB PRO/1000 MT Dual Port Server Adapter
    * proxy_server:
          o OS: Linux 2.6.21.5-smp i686 (Slackware 12.0)
          o CPU:  Intel(R) Pentium(R) III CPU family 1133MHz GenuineIntel
          o MEM: 1024M
          o DISK: SEAGATE ST318406LC SCSI Disk
          o Ethernet controller:  Intel 82557/8/9 PRO/100+ Server Adapter
    * client
          o OS: Linux 2.4.31 i686 (Slackware 10.2)
          o CPU: Intel(R) Xeon(TM) CPU 2.80GHz x 2
          o MEM: 1024M x 6
          o Ethernet controller: Intel 82546EB Gigabit Ethernet Controller
</pre>
<p><strong>file pool</strong></p>
<blockquote><p>
<i><br />
#!/bin/sh<br />
#<br />
# Written for generating 2 sets of 100Mbyte files,<br />
# one is 1,000 files of 100kbyte size, and<br />
# the other is 10 files of 10MByte size.<br />
# by Cherife Li <cherife@dotimes.com><br />
#</p>
<p>docroot=/home/wwwroot/</p>
<p># generate the first set<br />
mkdir -p $docroot/100k<br />
cd $docroot/100k<br />
for i in `seq 1 10`; do<br />
   mkdir -p files-$i;<br />
   for j in `seq 1 100`; do<br />
      dd if=/dev/zero of=files-$i/$j bs=100K count=1 2> /dev/null;<br />
   done;<br />
done</p>
<p># generate the second set<br />
mkdir -p $docroot/10m<br />
cd $docroot/10m<br />
for i in `seq 1 5`; do<br />
   mkdir -p files-$i;<br />
   for j in `seq 1 2`; do<br />
      dd if=/dev/zero of=files-$i/$j bs=10M count=1 2> /dev/null;<br />
   done;<br />
done<br />
</i></p></blockquote>
<p>$ <code>cd /home/wwwroot/</code><br />
$ <code>find ./100k/ | grep 'files.*/.' | sed 's#./#http://benchmark.lo/#' > 100k.urls</code><br />
$ <code>find ./10m/ | grep 'files.*/.' | sed 's#./#http://benchmark.lo/#' > 10m.urls</code></p>
<p><strong>benchmark</strong></p>
<pre>
 ______        ____________        ___________
|      |--A-->|            |--B-->|           |
|client|      |proxy server|      |http server|
|______|<--D--|____________|<--C--|___________|

    * client: http_load
    * proxy server: varnish 1.1.2, squid 2.6.STABLE18, and squid 3.0.STABLE2.
    * http server: nginx/0.6.28
</pre>
<p>The Nginx http server only ran the stand alone http service.<br />
The proxy server ran the proxy service only the one been benchmarking at a time.</p>
<p><strong>http_load</strong><br />
It&#8217;s a good load-generator as it</p>
<pre>
    * allows random fetches from a list of URLs
    * allows a large number of parallel requests
    * is portable.
</pre>
<p>There are also other test tools, check <a href="http://www.softwareqatest.com/qatweb1.html#LOAD" target="_blank" rel="nofollow">this page</a> for detail.<br />
<br />
+++ 100KByte +++<br />
<br />
$ <code>http_load -verbose -parallel 100 -fetches 100000 ./100k.urls</code><br />
<br />
* Varnish 1.1.2:</p>
<blockquote><p>
    &#8212; 60 secs, 6868 fetches started, 6768 completed, 100 current<br />
    &#8212; 120 secs, 13720 fetches started, 13620 completed, 100 current<br />
    &#8212; 180 secs, 20570 fetches started, 20470 completed, 100 current<br />
    &#8212; 240 secs, 27432 fetches started, 27332 completed, 100 current<br />
    &#8212; 300 secs, 34285 fetches started, 34185 completed, 100 current<br />
    &#8212; 360 secs, 41140 fetches started, 41040 completed, 100 current<br />
    &#8212; 420 secs, 47996 fetches started, 47896 completed, 100 current<br />
    &#8212; 480 secs, 54854 fetches started, 54754 completed, 100 current<br />
    &#8212; 540 secs, 61709 fetches started, 61609 completed, 100 current<br />
    &#8212; 600 secs, 68565 fetches started, 68465 completed, 100 current<br />
    &#8212; 660 secs, 75419 fetches started, 75319 completed, 100 current<br />
    &#8212; 720 secs, 82279 fetches started, 82179 completed, 100 current<br />
    &#8212; 780 secs, 89131 fetches started, 89031 completed, 100 current<br />
    &#8212; 840 secs, 95989 fetches started, 95889 completed, 100 current<br />
    100000 fetches, 100 max parallel, 1.024e+10 bytes, in 876.794 seconds<br />
    102400 mean bytes/connection<br />
    114.052 fetches/sec, 1.16789e+07 bytes/sec<br />
    msecs/connect: 116.758 mean, 9111.16 max, 0.256 min<br />
    msecs/first-response: 120.03 mean, 2494.66 max, 0.614 min<br />
    HTTP response codes:<br />
      code 200 &#8212; 100000
</p></blockquote>
<p>* Squid 2.6.STABLE18:</p>
<blockquote><p>
    &#8212; 60 secs, 6898 fetches started, 6798 completed, 100 current<br />
    &#8212; 120 secs, 13748 fetches started, 13648 completed, 100 current<br />
    &#8212; 180 secs, 20595 fetches started, 20495 completed, 100 current<br />
    &#8212; 240 secs, 27439 fetches started, 27339 completed, 100 current<br />
    &#8212; 300 secs, 34287 fetches started, 34187 completed, 100 current<br />
    &#8212; 360 secs, 41136 fetches started, 41036 completed, 100 current<br />
    &#8212; 420 secs, 47983 fetches started, 47883 completed, 100 current<br />
    &#8212; 480 secs, 54829 fetches started, 54729 completed, 100 current<br />
    &#8212; 540 secs, 61675 fetches started, 61575 completed, 100 current<br />
    &#8212; 600 secs, 68523 fetches started, 68423 completed, 100 current<br />
    &#8212; 660 secs, 75371 fetches started, 75271 completed, 100 current<br />
    &#8212; 720 secs, 82221 fetches started, 82121 completed, 100 current<br />
    &#8212; 780 secs, 89065 fetches started, 88965 completed, 100 current<br />
    &#8212; 840 secs, 95909 fetches started, 95809 completed, 100 current<br />
    100000 fetches, 100 max parallel, 1.024e+10 bytes, in 878.411 seconds<br />
    102400 mean bytes/connection<br />
    113.842 fetches/sec, 1.16574e+07 bytes/sec<br />
    msecs/connect: 116.02 mean, 9114.65 max, 0.224 min<br />
    msecs/first-response: 115.596 mean, 619.381 max, 0.86 min<br />
    HTTP response codes:<br />
      code 200 &#8212; 100000
</p></blockquote>
<p>* Squid 3.0.STABLE2:</p>
<blockquote><p>
    &#8212; 60 secs, 6885 fetches started, 6785 completed, 100 current<br />
    &#8212; 120 secs, 13720 fetches started, 13620 completed, 100 current<br />
    &#8212; 180 secs, 20577 fetches started, 20477 completed, 100 current<br />
    &#8212; 240 secs, 27418 fetches started, 27318 completed, 100 current<br />
    &#8212; 300 secs, 34266 fetches started, 34166 completed, 100 current<br />
    &#8212; 360 secs, 41111 fetches started, 41011 completed, 100 current<br />
    &#8212; 420 secs, 47957 fetches started, 47857 completed, 100 current<br />
    &#8212; 480 secs, 54812 fetches started, 54712 completed, 100 current<br />
    &#8212; 540 secs, 61656 fetches started, 61556 completed, 100 current<br />
    &#8212; 600 secs, 68503 fetches started, 68403 completed, 100 current<br />
    &#8212; 660 secs, 75346 fetches started, 75246 completed, 100 current<br />
    &#8212; 720 secs, 82198 fetches started, 82098 completed, 100 current<br />
    &#8212; 780 secs, 89040 fetches started, 88940 completed, 100 current<br />
    &#8212; 840 secs, 95891 fetches started, 95791 completed, 100 current<br />
    100000 fetches, 100 max parallel, 1.024e+10 bytes, in 876.658 seconds<br />
    102400 mean bytes/connection<br />
    114.07 fetches/sec, 1.16807e+07 bytes/sec<br />
    msecs/connect: 115.858 mean, 9111.16 max, 0.26 min<br />
    msecs/first-response: 116.423 mean, 3318.18 max, 29.481 min<br />
    HTTP response codes:<br />
      code 200 &#8212; 100000
</p></blockquote>
<p>+++ 10MByte +++<br />
<br />
$ <code>http_load -verbose -parallel 100 -fetches 1000 ./10m.urls</code><br />
<br />
* Varnish 1.1.2:</p>
<blockquote><p>
    &#8212; 60 secs, 100 fetches started, 0 completed, 100 current<br />
    &#8212; 120 secs, 196 fetches started, 96 completed, 100 current<br />
    &#8212; 180 secs, 231 fetches started, 131 completed, 100 current<br />
    &#8212; 240 secs, 304 fetches started, 204 completed, 100 current<br />
    &#8212; 300 secs, 389 fetches started, 289 completed, 100 current<br />
    &#8212; 360 secs, 434 fetches started, 334 completed, 100 current<br />
    &#8212; 420 secs, 509 fetches started, 409 completed, 100 current<br />
    &#8212; 480 secs, 585 fetches started, 485 completed, 100 current<br />
    &#8212; 540 secs, 637 fetches started, 537 completed, 100 current<br />
    &#8212; 600 secs, 714 fetches started, 614 completed, 100 current<br />
    &#8212; 660 secs, 786 fetches started, 686 completed, 100 current<br />
    &#8212; 720 secs, 844 fetches started, 744 completed, 100 current<br />
    &#8212; 780 secs, 915 fetches started, 815 completed, 100 current<br />
    &#8212; 840 secs, 987 fetches started, 887 completed, 100 current<br />
    1000 fetches, 100 max parallel, 1.04858e+10 bytes, in 899.718 seconds<br />
    1.04858e+07 mean bytes/connection<br />
    1.11146 fetches/sec, 1.16545e+07 bytes/sec<br />
    msecs/connect: 129.428 mean, 3128.09 max, 0.291 min<br />
    msecs/first-response: 1003.12 mean, 9619.7 max, 120.662 min<br />
    HTTP response codes:<br />
      code 200 &#8212; 1000
</p></blockquote>
<p>* Squid 2.6.STABLE18:</p>
<blockquote><p>
    &#8212; 60 secs, 103 fetches started, 3 completed, 100 current<br />
    &#8212; 120 secs, 178 fetches started, 78 completed, 100 current<br />
    &#8212; 180 secs, 255 fetches started, 155 completed, 100 current<br />
    &#8212; 240 secs, 322 fetches started, 222 completed, 100 current<br />
    &#8212; 300 secs, 379 fetches started, 279 completed, 100 current<br />
    &#8212; 360 secs, 458 fetches started, 358 completed, 100 current<br />
    &#8212; 420 secs, 518 fetches started, 418 completed, 100 current<br />
    &#8212; 480 secs, 583 fetches started, 483 completed, 100 current<br />
    &#8212; 540 secs, 661 fetches started, 561 completed, 100 current<br />
    &#8212; 600 secs, 721 fetches started, 621 completed, 100 current<br />
    &#8212; 660 secs, 786 fetches started, 686 completed, 100 current<br />
    &#8212; 720 secs, 863 fetches started, 763 completed, 100 current<br />
    &#8212; 780 secs, 926 fetches started, 826 completed, 100 current<br />
    &#8212; 840 secs, 984 fetches started, 884 completed, 100 current<br />
    1000 fetches, 100 max parallel, 1.04858e+10 bytes, in 894.062 seconds<br />
    1.04858e+07 mean bytes/connection<br />
    1.11849 fetches/sec, 1.17282e+07 bytes/sec<br />
    msecs/connect: 137.644 mean, 9128.16 max, 0.283 min<br />
    msecs/first-response: 131.811 mean, 3509.99 max, 28.891 min<br />
    HTTP response codes:<br />
      code 200 &#8212; 1000
</p></blockquote>
<p>* Squid 3.0.STABLE2:</p>
<blockquote><p>
    &#8212; 60 secs, 105 fetches started, 5 completed, 100 current<br />
    &#8212; 120 secs, 190 fetches started, 90 completed, 100 current<br />
    http://benchmark.lo/10m/files-2/2: timed out<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-1/1: timed out<br />
    http://benchmark.lo/10m/files-1/1: byte count wrong<br />
    http://benchmark.lo/10m/files-5/2: timed out<br />
    http://benchmark.lo/10m/files-5/2: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-1/1: byte count wrong<br />
    http://benchmark.lo/10m/files-2/1: timed out<br />
    http://benchmark.lo/10m/files-2/1: byte count wrong<br />
    http://benchmark.lo/10m/files-1/1: byte count wrong<br />
    http://benchmark.lo/10m/files-2/1: byte count wrong<br />
    http://benchmark.lo/10m/files-2/1: byte count wrong<br />
    http://benchmark.lo/10m/files-4/2: timed out<br />
    http://benchmark.lo/10m/files-4/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-4/2: byte count wrong<br />
    http://benchmark.lo/10m/files-4/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-4/2: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-1/1: byte count wrong<br />
    http://benchmark.lo/10m/files-1/1: byte count wrong<br />
    http://benchmark.lo/10m/files-4/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/1: byte count wrong<br />
    http://benchmark.lo/10m/files-2/1: byte count wrong<br />
    http://benchmark.lo/10m/files-1/1: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/1: byte count wrong<br />
    http://benchmark.lo/10m/files-4/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-4/2: byte count wrong<br />
    &#8212; 180 secs, 260 fetches started, 160 completed, 100 current<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-1/1: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/1: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-3/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/1: byte count wrong<br />
    http://benchmark.lo/10m/files-4/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    http://benchmark.lo/10m/files-2/2: byte count wrong<br />
    &#8212; 240 secs, 315 fetches started, 215 completed, 100 current<br />
    &#8212; 300 secs, 395 fetches started, 295 completed, 100 current<br />
    &#8212; 360 secs, 464 fetches started, 364 completed, 100 current<br />
    &#8212; 420 secs, 521 fetches started, 421 completed, 100 current<br />
    &#8212; 480 secs, 596 fetches started, 496 completed, 100 current<br />
    &#8212; 540 secs, 664 fetches started, 564 completed, 100 current<br />
    &#8212; 600 secs, 725 fetches started, 625 completed, 100 current<br />
    &#8212; 660 secs, 798 fetches started, 698 completed, 100 current<br />
    &#8212; 720 secs, 865 fetches started, 765 completed, 100 current<br />
    &#8212; 780 secs, 929 fetches started, 829 completed, 100 current<br />
    &#8212; 840 secs, 997 fetches started, 897 completed, 100 current<br />
    1000 fetches, 100 max parallel, 1.03922e+10 bytes, in 886.272 seconds<br />
    1.03922e+07 mean bytes/connection<br />
    1.12832 fetches/sec, 1.17258e+07 bytes/sec<br />
    msecs/connect: 130.639 mean, 3127.72 max, 0.21 min<br />
    msecs/first-response: 186.15 mean, 3578.41 max, 34.601 min<br />
    5 timeouts<br />
    53 bad byte counts<br />
    HTTP response codes:<br />
      code 200 &#8212; 1000
</p></blockquote>
<p><strong>proxy server status</strong></p>
<p>+++ 100KByte +++</p>
<p><a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/100kvarnishiostat_xm5.log' title='varnish.iostat_xm5.log'>100k.varnish.iostat_xm5.log</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/100kvarnishvmstat5.log' title='varnish.vmstat5.log'>100k.varnish.vmstat5.log</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/100ksquid2iostat_xm5.log' title='squid2.iostat_xm5.log'>100k.squid2.iostat_xm5.log</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/100ksquid2vmstat5.log' title='squid2.vmstat5.log'>100k.squid2.vmstat5.log</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/100ksquid3iostat_xm5.log' title='squid3.iostat_xm5.log'>100k.squid3.iostat_xm5.log</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/100ksquid3vmstat5.log' title='squid3.vmstat5.log'>100k.squid3.vmstat5.log</a></p>
<p>+++ 10MByte +++</p>
<p><a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/10mvarnishiostat_xm5.log' title='varnish.iostat_xm5.log'>10m.varnish.iostat_xm5.log</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/10mvarnishvmstat5.log' title='10m.varnish.vmstat5.log'>10m.varnish.vmstat5.log</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/10msquid2iostat_xm5.log' title='10m.squid2.iostat_xm5.log'>10m.squid2.iostat_xm5.log</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/10msquid2vmstat5.log' title='10m.squid2.vmstat5.log'>10m.squid2.vmstat5.log</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/10msquid3iostat_xm5.log' title='10m.squid3.iostat_xm5.log'>10m.squid3.iostat_xm5.log</a><br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/10msquid3vmstat5.log' title='10m.squid3.vmstat5.log'>10m.squid3.vmstat5.log</a></p>
<p>After finished each benchmark, the status is as follow:<br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/bench_varnishstat.png' title='varnishstat'><img src='http://dotimes.com/iscale/wp-content/uploads/2008/03/bench_varnishstat.thumbnail.png' alt='varnishstat' /></a>    <a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/bench_squid2_mgr_info.png' title='squid2_mgr_info'><img src='http://dotimes.com/iscale/wp-content/uploads/2008/03/bench_squid2_mgr_info.thumbnail.png' alt='squid2_mgr_info' /></a>    <a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/bench_squid3_mgr_info.png' title='squid3_mgr_info'><img src='http://dotimes.com/iscale/wp-content/uploads/2008/03/bench_squid3_mgr_info.thumbnail.png' alt='squid3_mgr_info' /></a></p>
<p><strong>conclusion</strong></p>
<p>The data here, the reference in your mind.<br />
That which is the better choice as a caching proxy, depends on your own judgement.</p>
<p><strong>notes</strong></p>
<p>1. Varnish can cache files immediately.<br />
I found this while comparing the fetching of 10MB files.<br />
Varnish just request the nginx server for the file only once, but Squid will requests them for about<br />
10 times of varnish does.</p>
<p>2. Varnish provides easier access to proxy status.</p>
<p>3. Squid is more than a web proxy, while Varnish is more than an http accelerator (cache proxy).<br />
The Varnish web site claims that Varnish is ten to twenty times faster than the popular Squid<br />
cache on the same hardware.</p>
]]></content:encoded>
			<wfw:commentRss>http://dotimes.com/iscale/2008/03/benchmark-caching-of-varnish-and-squid.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>FUSE - Filesystem in Userspace</title>
		<link>http://dotimes.com/iscale/2008/03/fuse-filesystem-in-userspace.html</link>
		<comments>http://dotimes.com/iscale/2008/03/fuse-filesystem-in-userspace.html#comments</comments>
		<pubDate>Sun, 09 Mar 2008 12:13:57 +0000</pubDate>
		<dc:creator>Cherife Li</dc:creator>
		
		<category><![CDATA[linux]]></category>

		<category><![CDATA[storage]]></category>

		<category><![CDATA[filesystem]]></category>

		<category><![CDATA[slackware]]></category>

		<guid isPermaLink="false">http://dotimes.com/iscale/2008/03/fuse-filesystem-in-userspace.html</guid>
		<description><![CDATA[Yesterday, Slackware -current repo got a big update &#8212; mainly kernel related stuff. Besides, Pat moved ntfsprogs tgz from /extra to ap/, which now includes full read-write support when the NTFS filesystem is mounted with ntfsmount or -t fuse.ntfs. This feature requires FUSE, which is added by Pat too.
Now we Slackers won&#8217;t need to do [...]]]></description>
			<content:encoded><![CDATA[<p>Yesterday, Slackware -current repo got a big update &#8212; mainly kernel related stuff. Besides, Pat moved ntfsprogs tgz from /extra to ap/, which now includes full read-write support when the NTFS filesystem is mounted with ntfsmount or -t fuse.ntfs. This feature requires FUSE, which is added by Pat too.<br />
Now we Slackers won&#8217;t need to do much superfluous things ourselves to implementation full read-write support feature. I&#8217;m just a bit lazy, sometimes :-)</p>
<p>An simple intro from the <a href="http://fuse.sourceforge.net/" target="_blank" title="FUSE: Filesystem in Userspace">FUSE website</a>:</p>
<pre>
With FUSE it is possible to implement a fully functional filesystem in a userspace program.  Features include:

    * Simple library API
    * Simple installation (no need to patch or recompile the kernel)
    * Secure implementation
    * Userspace - kernel interface is very efficient
    * Usable by non privileged users
    * Runs on Linux kernels 2.4.X and 2.6.X
    * Has proven very stable over time
</pre>
<p>The following figure shows the path of a filesystem call:<br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2008/03/fuse_structure.png' title='fuse structure'><img src='http://dotimes.com/iscale/wp-content/uploads/2008/03/fuse_structure.png' alt='fuse structure' /></a></p>
<p>Operating Systems FUSE Supportes are Linux-2.4.X, Linux-2.6.X, FreeBSD, NetBSD, Mac OS X, Windows, OpenSolaris, GNU/Hurd. Check <a href="http://fuse.sourceforge.net/wiki/index.php/OperatingSystems" target="_blank" title="Detailed Operating Systems FUSE Supports">this link</a> for detail.</p>
<p>There are various types of Filesystems based on FUSE, like:</p>
<pre>
# ArchiveFileSystems - accessing files inside archives (tar, cpio, zip, etc.)
# CompressedFileSystems - accessing files in a compressed image (gz, zlib, LiveCDs, etc.)
# DatabaseFileSystems - storing files in a relational database (MySQL, BerkeleyDB, etc.) \
    or ones allowing searching using tags or SQL queries
# EncryptedFileSystems - storing files in a more secure way by using a secret key
# MediaFileSystems - storing files on media devices such as cameras and music players \
    or accessing and categorizing media files
# HardwareFileSystems - provide access to weird hardware
# MonitoringFileSystems - provide notification when a file changes
# NetworkFileSystems - storing files on remote computers, including file servers and web sites
# NonNativeFileSystems - traditional disk-based file systems that aren't standard on \
    Linux (NTFS, ZFS, etc.)
# UnionFileSystems - merging multiple file systems into a single tree
# VersioningFileSystems - file systems that remember old versions of files and ones which \
    provide access to version control systems
</pre>
<p>Check <a href="http://fuse.sourceforge.net/wiki/index.php/FileSystems" target="_blank" title="Filesystems based on FUSE">this link</a> for up-to-date detail.</p>
]]></content:encoded>
			<wfw:commentRss>http://dotimes.com/iscale/2008/03/fuse-filesystem-in-userspace.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>SystemImager - Make Things Automatically Done</title>
		<link>http://dotimes.com/iscale/2008/03/systemimager-make-things-automatically-done.html</link>
		<comments>http://dotimes.com/iscale/2008/03/systemimager-make-things-automatically-done.html#comments</comments>
		<pubDate>Sun, 02 Mar 2008 14:10:37 +0000</pubDate>
		<dc:creator>Cherife Li</dc:creator>
		
		<category><![CDATA[linux]]></category>

		<category><![CDATA[systemimager]]></category>

		<guid isPermaLink="false">http://dotimes.com/iscale/2008/03/systemimager-make-things-automatically-done.html</guid>
		<description><![CDATA[Intro:
SystemImager is a part of System Installation Suite which automates Linux installs, software distribution, and production deployment.
SystemImager makes it easy to do automated installs (clones), software distribution, content or data distribution, configuration changes, and operating system updates to your network of Linux machines. You can even update from one Linux release version to another!
It can [...]]]></description>
			<content:encoded><![CDATA[<p><strong>Intro:</strong></p>
<p>SystemImager is a part of System Installation Suite which automates Linux installs, software distribution, and production deployment.</p>
<p>SystemImager makes it easy to do automated installs (clones), software distribution, content or data distribution, configuration changes, and operating system updates to your network of Linux machines. You can even update from one Linux release version to another!</p>
<p>It can also be used to ensure safe production deployments. By saving your current production image before updating to your new production image, you have a highly reliable contingency mechanism. If the new production enviroment is found to be flawed, simply roll-back to the last production image with a simple update command!</p>
<p>Some typical environments include: Internet server farms, database server farms, high performance clusters, computer labs, and corporate desktop environments. </p>
<p><strong>Benchmark</strong></p>
<p>Check <a href="http://wiki.systemimager.org/index.php/BitTorrent#Benchmark" title="a benchmark of systemimager - Installing more than 1000 nodes in 15min with SystemImager" target="_blank" rel="nofollow">this link</a> for detail.</p>
<p><strong>Useful Resource</strong></p>
<pre>
    * Article on linux.com: <a href="http://www.linux.com/article.pl?sid=06/04/24/1724216" target="_blank" rel="nofollow">Automate Linux installation and recovery with SystemImager</a>
    * HowTo sur Capweb.biz : <a href="http://www.capweb.biz/?page=tutoriaux&#038;voir=articles/linux/backup/image_backup_with_systemimager" target="_blank" rel="nofollow">Cr茅er des images de votre syst猫me Linux avec SystemImager</a> (FR)
    * <a href="http://www.linuxjournal.com/article/9045" target="_blank" rel="nofollow">Clustering Is Not Rocket Science</a>
    * <a href="http://www.gridtoday.com/grid/720347.html" target="_blank" rel="nofollow">Avatars and Grid 2.0</a> (article on Grid Today quoting SystemImager as a core component \
        used in the famous virtual world Second Life)
    * <a href="http://howto.krisbuytaert.be/AutomatingVirtualMachineDeployment" target="_blank" rel="nofollow">Automating Xen Virtual Machine Deployment</a> (how to integrate SystemImager with Xen to \
        create a totally open source deployment framework of virtual machines) 

    * <a href="http://www.nostarch.com/frameset.php?startat=cluster" target="_blank" rel="nofollow">The Linux Enterprise Cluster</a>
    * <a href="http://www.oreilly.com/catalog/highperlinuxc/" target="_blank" rel="nofollow">High Performance Linux Clusters with OSCAR, Rocks, OpenMosix, and MPI</a>
    * <a href="http://www.samspublishing.com/bookstore/product.asp?isbn=1578702747&#038;aid=E9AA1AB7-0303-47AB-9F3E-775CF68B45AC&#038;rl=1" target="_blank" rel="nofollow">Linux Clustering: Building and Maintaining Linux Clusters</a>
    * <a href="http://www.springer.com/west/home?SGWID=4-102-22-33313559-0&#038;changeHeader=true&#038;referer=www.springeronline.com&#038;SHORTCUT=www.springer.com/west/0-7923-7892-X" target="_blank" rel="nofollow">Distributed and Parallel Systems: From Instruction Parallelism to Cluster Computing</a>
    * <a href="http://www.intel.com/cd/ids/developer/asmo-na/eng/102952.htm" target="_blank" rel="nofollow">High Performance Computing Clusters with Intel庐 Architecture, Part 2 </a>

    * <a href="http://dague.net/sean/sispaper.pdf" target="_blank" rel="nofollow">System Installation Suite: Massive Installation for Linux</a> - Sean Dague

    * <a href="http://sisuite.org/presentations/OLS_2002/sisols.pdf" target="_blank" rel="nofollow">SystemImager @ Ottawa Linux Symposium 2002</a> - Sean Dague
    * <a href="http://www.linuxtag.org/2007/en/conf/events/vp-donnerstag/details-talkid-63.html" target="_blank" rel="nofollow">SystemImager @ LinuxTag 2007 in Berlin</a> - Andrea Righi, Brian Elliott Finley, Erich Focht
          o Get <a href="http://download.systemimager.org/pub/docs/SystemImager-LinuxTag-2007-paper.pdf" target="_blank" rel="nofollow">paper</a> and <a href="http://download.systemimager.org/pub/docs/SystemImager-LinuxTag-2007-presentation.pdf" target="_blank" rel="nofollow">presentation</a> of the talk in PDF 

    * <a href="http://www.cineca.it/bdp/sezioni/risorse/hardware/scheda?CODICE=bcx_5120_hpc&#038;BSDVAR_TIPOLOGIA=hpcspeciali" target="_blank" rel="nofollow">SystemImager at CINECA</a>
          o The IBM BCX/5120 cluster in CINECA uses SystemImager
    * <a href="https://wiki.fysik.dtu.dk/niflheim/SystemImager_Installation" target="_blank" rel="nofollow">SystemImager at niflheim</a>
          o Wiki about SystemImager used for the NIFLHEIM cluster installation
</pre>
]]></content:encoded>
			<wfw:commentRss>http://dotimes.com/iscale/2008/03/systemimager-make-things-automatically-done.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>inotify - monitoring file system events</title>
		<link>http://dotimes.com/iscale/2007/12/inotify-monitoring-file-system-events.html</link>
		<comments>http://dotimes.com/iscale/2007/12/inotify-monitoring-file-system-events.html#comments</comments>
		<pubDate>Sat, 15 Dec 2007 09:36:09 +0000</pubDate>
		<dc:creator>Cherife Li</dc:creator>
		
		<category><![CDATA[linux]]></category>

		<category><![CDATA[filesystem]]></category>

		<guid isPermaLink="false">http://dotimes.com/iscale/2007/12/inotify-monitoring-file-system-events.html</guid>
		<description><![CDATA[If you have some experience on compiling linux kernel, maybe you could recall that there is an option called &#8220;Inotify file change notification support&#8221; under &#8220;File Systems&#8221; section. Inotify is an inode-based filesystem notification technology. It provides possibility to simply monitor various events on files in filesystems. It is a very much powerful replacement of [...]]]></description>
			<content:encoded><![CDATA[<p>If you have some experience on compiling linux kernel, maybe you could recall that there is an option called <em>&#8220;Inotify file change notification support&#8221;</em> under <em>&#8220;File Systems&#8221;</em> section. <a href="http://en.wikipedia.org/wiki/Inotify" rel="nofollow">Inotify</a> is an inode-based filesystem notification technology. It provides possibility to simply monitor various events on files in filesystems. It is a very much powerful replacement of (obsolete) dnotify. Check <a href="http://www.kernel.org/pub/linux/kernel/people/rml/inotify/README">Why Not dnotify and Why inotify</a> to see a brief comparison between dnotify and inotify from <a href="http://rlove.org/">Robert Love</a>.</p>
<p>In kernel space, the inotify API provides a mechanism for monitoring file system events. Inotify can be used to monitor individual files, or to monitor directories. When a directory is monitored, inotify will return events for the directory itself, and for files inside the directory.</p>
<p>
In user space, there are several tools use inotify, such as <a href="http://pyinotify.sourceforge.net/">pyinotify</a>, <a href="http://sourceforge.net/projects/iwatch">IWatch</a>, <a href="http://incron.aiken.cz/">incron</a>, <a href="http://logsend.sourceforge.net/">logsend</a>, <a href="http://mswatch.sourceforge.net/">mswatch</a>, <a href="http://distanz.ch/inotail/">inotail</a>, <a href="http://inotify-tools.sourceforge.net/">inotify-tools</a> and the others I don&#8217;t know of.<br />
On Dec 3, inotify-tools was added into Slackware-Current repository. As the style of  Patrick Volkerding and Slackware Distro, I think inotify-tools is worthy more to use.</p>
<p>Inotify-tools currently consists of a library called &#8220;<em>libinotifytools</em>&#8221; and two programs called &#8220;<em>inotifywait</em>&#8221; and &#8220;<em>inotifywatch</em>&#8220;, with a third kinda sorta planned but don&#8217;t hold your breath.</p>
<p><em>Libinotifytools</em> library provides a thin layer on top of the basic inotify interface. The primary use is to easily set up watches on files, potentially many files at once, and read events without having to deal with low-level I/O. There are also several utility functions for inotify-related string formatting.</p>
<p><em>Inotifywait</em> efficiently waits for changes to files using Linux&#8217;s inotify interface. It is suitable for waiting for changes to files from shell scripts. It can either exit once an event occurs, or continually execute and output events as they occur, and can recursively watch entire directory trees.</p>
<p><em>Inotifywatch</em> listens for filesystem events using Linux&#8217;s inotify interface, then outputs a summary count of the events received on each file or directory.</p>
<p>How to use them? Please check their man pages for further information :-)</p>
<p>Below is a test on inotifywait. On one console monitored outputs, on another did some operation.</p>
<p>monitoring console:</p>
<pre>
root@ws:~# inotifywait -mr --format '%w %f %e' /home/cherife
Setting up watches.
Beware: since -r was given, this may take a while!
Watches established.
/home/cherife/tmp/ test CREATE,ISDIR
/home/cherife/tmp/ test OPEN,ISDIR
/home/cherife/tmp/ test CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/ test OPEN,ISDIR
/home/cherife/tmp/test/  OPEN,ISDIR
/home/cherife/tmp/ test CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/  CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/ new CREATE,ISDIR
/home/cherife/tmp/test/ new OPEN,ISDIR
/home/cherife/tmp/test/ new CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/  OPEN,ISDIR
/home/cherife/  CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/ tmp OPEN,ISDIR
/home/cherife/tmp/  OPEN,ISDIR
/home/cherife/ tmp CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/  CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/ test OPEN,ISDIR
/home/cherife/tmp/test/  OPEN,ISDIR
/home/cherife/tmp/ test CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/  CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/new/ foo CREATE
/home/cherife/tmp/test/new/ foo OPEN
/home/cherife/tmp/test/new/ foo MODIFY
/home/cherife/tmp/test/new/ foo CLOSE_WRITE,CLOSE
/home/cherife/  OPEN,ISDIR
/home/cherife/  CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/ tmp OPEN,ISDIR
/home/cherife/tmp/  OPEN,ISDIR
/home/cherife/ tmp CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/  CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/ test OPEN,ISDIR
/home/cherife/tmp/test/  OPEN,ISDIR
/home/cherife/tmp/ test CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/  CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/ test OPEN,ISDIR
/home/cherife/tmp/test/  OPEN,ISDIR
/home/cherife/tmp/ test CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/  CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/ new OPEN,ISDIR
/home/cherife/tmp/test/new/  OPEN,ISDIR
/home/cherife/tmp/test/ new CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/new/  CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/new/ foo OPEN
/home/cherife/tmp/test/new/ foo ACCESS
/home/cherife/tmp/test/new/ foo CLOSE_NOWRITE,CLOSE
/home/cherife/tmp/ test OPEN,ISDIR
/home/cherife/tmp/test/  OPEN,ISDIR
/home/cherife/tmp/ test CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/  CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/ new OPEN,ISDIR
/home/cherife/tmp/test/new/  OPEN,ISDIR
/home/cherife/tmp/test/new/ foo DELETE
/home/cherife/tmp/test/ new CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/new/  CLOSE_NOWRITE,CLOSE,ISDIR
/home/cherife/tmp/test/ new DELETE,ISDIR
/home/cherife/tmp/test/new/  DELETE_SELF
/home/cherife/tmp/test/new/  IGNORED
</pre>
<p>operations:</p>
<pre>
[cherife@ws:~/tmp/test $] history|tail -6
520  mkdir -p tmp/test/new
521  echo "this is a test." &gt; tmp/test/new/foo
522  cd tmp/test
523  cat new/foo
524  rm new/ -r
525  history|tail -6
</pre>
<p>In some case, yeah, it&#8217;s very powerful and useful for system administration.</p>
]]></content:encoded>
			<wfw:commentRss>http://dotimes.com/iscale/2007/12/inotify-monitoring-file-system-events.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>Packaging Varnish for Slackware</title>
		<link>http://dotimes.com/iscale/2007/12/packaging-varnish-for-slackware.html</link>
		<comments>http://dotimes.com/iscale/2007/12/packaging-varnish-for-slackware.html#comments</comments>
		<pubDate>Thu, 13 Dec 2007 16:03:12 +0000</pubDate>
		<dc:creator>Cherife Li</dc:creator>
		
		<category><![CDATA[linux]]></category>

		<category><![CDATA[slackware]]></category>

		<category><![CDATA[varnish]]></category>

		<guid isPermaLink="false">http://dotimes.com/iscale/2007/12/packaging-varnish-for-slackware.html</guid>
		<description><![CDATA[It&#8217;s always necessary and a good manner to use the system package management tools when installing software into the system, rather than just issuing &#8220;./configure &#038;&#038; make &#038;&#038; make install&#8221; which would lead the file system to be knacker&#8217;s yard.
Okay, maybe you need to read about slackware package howtos before continuing reading ;)
The related files [...]]]></description>
			<content:encoded><![CDATA[<p>It&#8217;s always necessary and a good manner to use the system package management tools when installing software into the system, rather than just issuing &#8220;./configure &#038;&#038; make &#038;&#038; make install&#8221; which would lead the file system to be knacker&#8217;s yard.</p>
<p>Okay, maybe you need to read about <a href="http://www.google.com/search?q=slackware+package+howto" target="_blank" title="Googling 'slackware+package+howto'">slackware package howtos</a> before continuing reading ;)</p>
<p>The related files are as below. You&#8217;d better use the newest ones in <a href="http://dotimes.com/slackbuilds/varnish/">my SlackBuild directory</a>.</p>
<p><a href='http://dotimes.com/iscale/wp-content/uploads/2007/12/slack-desc' title='slack-desc'>slack-desc</a></p>
<pre>
varnish: VARNISH - a high-performance HTTP accelerator
varnish:
varnish: Varnish is targeted primarily at the FreeBSD 6 and
varnish: Linux 2.6 platforms, and will take full advantage
varnish: of the virtual memory system and advanced I/O
varnish: features offered by these operating systems.
varnish:
varnish: Homepage: http://varnish.projects.linpro.no/
varnish:
varnish:
</pre>
<p><a href='http://dotimes.com/iscale/wp-content/uploads/2007/12/rc.varnishd' title='varnish init script for slackware'>rc.varnishd</a></p>
<pre>
#!/bin/sh
# Written to start/stop/restart varnishd.
# by Cherife Li &lt;cherife@dotimes.com&gt;
#

VARNISH_VCL_CONF=/etc/varnish/default.vcl
VARNISH_PIDFILE=/var/run/varnish.pid
VARNISH_LOCKFILE=/var/lock/subsys/varnish
VARNISH_DAEMON=/usr/sbin/varnishd
VARNISH_PARAM="-p thread_pool_max=1500 \
                                -p thread_pools=5 \
                                -p listen_depth=512 \
                                -p client_http11=off"

VARNISH_LOG_PIDFILE=/var/run/varnishlog.pid
VARNISH_LOG_LOCKFILE=/var/lock/subsys/varnishlog
VARNISH_LOG_DAEMON=/usr/bin/varnishlog
LOGFILE=/var/log/varnish.log
VARNISH_LOG_OPTS="-a -w ${LOGFILE} \
                                      -D -P ${VARNISH_LOG_PIDFILE}"

# Default address and port to bind to
# Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
VARNISH_LISTEN_ADDRESS=
VARNISH_LISTEN_PORT=80

# Telnet admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=81

# The minimum number of worker threads to start
VARNISH_MIN_THREADS=1

# The Maximum number of worker threads to start
VARNISH_MAX_THREADS=100

# Idle timeout for worker threads
VARNISH_THREAD_TIMEOUT=120

# Cache file location
VARNISH_STORAGE_FILE=/var/lib/varnish_storage.bin

# Cache file size: in bytes, optionally using k / M / G / T suffix,
# or in percentage of available disk space using the % suffix.
VARNISH_STORAGE_SIZE=1G

# Backend storage specification
VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},\
                                    ${VARNISH_STORAGE_SIZE}"

# Default TTL used when the backend does not specify one
VARNISH_TTL=120

# VARNISH_DAEMON_OPTS is used by the init script.
# If you add or remove options, make sure you
# update this section, too.
VARNISH_DAEMON_OPTS="\
-a ${VARNISH_LISTEN_ADDRESS}:\
     ${VARNISH_LISTEN_PORT} \
-f ${VARNISH_VCL_CONF} \
-T ${VARNISH_ADMIN_LISTEN_ADDRESS}:\
     ${VARNISH_ADMIN_LISTEN_PORT} \
-t ${VARNISH_TTL} \
-w ${VARNISH_MIN_THREADS},\
     ${VARNISH_MAX_THREADS},\
     ${VARNISH_THREAD_TIMEOUT} \
-u nobody -g nobody \
-P ${VARNISH_PIDFILE} \
-s ${VARNISH_STORAGE}"

case "$1" in
'start')
        echo -n "Starting varnish daemon..."
        $VARNISH_DAEMON \
                $VARNISH_DAEMON_OPTS \
                $VARNISH_PARAM > /dev/null 2>&#038;1
        $VARNISH_LOG_DAEMON $VARNISH_LOG_OPTS
        touch $VARNISH_LOCKFILE $VARNISH_LOG_LOCKFILE
        echo
        ;;
'stop')
        echo -n "Stopping varnish daemon..."
        kill -9 `cat $VARNISH_PIDFILE`
        kill -9 `cat $VARNISH_LOG_PIDFILE`
        rm -f $VARNISH_PIDFILE $VARNISH_LOG_PIDFILE \
                  $VARNISH_LOCKFILE $VARNISH_LOG_LOCKFILE
        killall varnishd 2> /dev/null
        echo
        ;;
'restart')
        $0 stop
        sleep 1
        $0 start
        ;;
*)
        echo "usage $0 start|stop|restart"
        exit 1
esac

exit 0
</pre>
<p><a href='http://dotimes.com/iscale/wp-content/uploads/2007/12/varnish.SlackBuild' title='slackware package build script for varnish'>varnish.SlackBuild</a></p>
<pre>
#!/bin/sh
# Build/install varnish the way Slackware's binary package is made:
# Written by Cherife Li &lt;cherife@dotimes.com&gt;

# Set initial variables:
PRGNAM=varnish
VERSION=1.1.2
ARCH=${ARCH:-i486}
BUILD=${BUILD:-1}
CWD=$(pwd)
TMP=${TMP:-/tmp/SBo}
PKG=$TMP/package-$PRGNAM
OUTPUT=${OUTPUT:-/tmp}

DOCUMENTATION="ChangeLog LICENSE INSTALL README"

if [ "$ARCH" = "i386" ]; then
  SLKCFLAGS="-O2 -march=i386 -mcpu=i686"
elif [ "$ARCH" = "i486" ]; then
  SLKCFLAGS="-O2 -march=i486 -mtune=i686"
elif [ "$ARCH" = "i686" ]; then
  SLKCFLAGS="-O2 -march=i686 -mtune=i686"
fi

set -e

rm -rf $PKG
mkdir -p $TMP $PKG $OUTPUT
cd $TMP
rm -rf $PRGNAM-$VERSION
tar -xvf $CWD/$PRGNAM-$VERSION.tar.gz
cd $PRGNAM-$VERSION
chown -R root:root .
chmod -R u+w,go+r-w,a-s .

CFLAGS="$SLKCFLAGS" \
CXXFLAGS="$SLKCFLAGS" \
./configure \
  --prefix=/usr \
  --sysconfdir=/etc/varnish/ \
  --localstatedir=/var \
  --mandir=/usr/man

make -j3 || exit 1
make install DESTDIR=$PKG || exit 1

mkdir -p $PKG/etc/{varnish,rc.d} $PKG/usr/man
cat $TMP/$PRGNAM-$VERSION/etc/default.vcl > $PKG/etc/default.vcl.new
cat $TMP/$PRGNAM-$VERSION/etc/zope-plone.vcl > $PKG/etc/zope-plone.vcl.new
cat $CWD/rc.varnishd > $PKG/etc/rc.d/rc.varnishd.new

# Strip everything for good measure:
( cd $PKG
  find . | xargs file | grep "executable" | grep ELF |\
        cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null
  find . | xargs file | grep "shared object" | grep ELF |\
        cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null
)

mkdir -p $PKG/usr/doc/$PRGNAM-$VERSION
cp -a $DOCUMENTATION $PKG/usr/doc/$PRGNAM-$VERSION
cat $CWD/$PRGNAM.SlackBuild > \
        $PKG/usr/doc/$PRGNAM-$VERSION/$PRGNAM.SlackBuild

( cd $PKG/usr/man
  find . -type f -exec gzip -9 {} \;
  for i in $( find . -type l ) ; do ln -s $( readlink $i ).gz $i.gz ; rm $i ; done
)

mkdir -p $PKG/install
cat $CWD/slack-desc > $PKG/install/slack-desc
cat $CWD/doinst.sh > $PKG/install/doinst.sh

cd $PKG
/sbin/makepkg -l y -c n \
        $OUTPUT/$PRGNAM-$VERSION-$ARCH-$BUILD$TAG.tgz
</pre>
<p>Tarball download<br />
<a href='http://dotimes.com/iscale/wp-content/uploads/2007/12/varnish.tar.gz' title='slackware packaging code'>Download a tarball</a> including the three files above, but without the varnish source code.</p>
<p>Enjoy (-:</p>
]]></content:encoded>
			<wfw:commentRss>http://dotimes.com/iscale/2007/12/packaging-varnish-for-slackware.html/feed</wfw:commentRss>
		</item>
	</channel>
</rss>
