Serving small static files: which server to use ?

Update 1 (Mar 29, 2011): Logging and compression now disabled for all servers
Update 2 (Apr 14, 2011): Added results for Nginx 1.0

Introduction

The goal of this benchmark is to compare several web server or caching server regarding their respective ability and performance to server small static files. The servers are optimized, contrary to my previous benchmark, where each server was tested using its default settings. I consider in this benchmark only the most performant open source servers, namely Varnish Cache, Nginx, Lighttpd, Apache Traffic Server as well as G-WAN (free, but not open source), as it was the clear winner of the previous test.

Serving static files: a comparison between Apache, Nginx, Varnish and G-WAN

Update 1 (Mar 16, 2011): Apache MPM-Event benchmark added
Update 2 (Mar 16, 2011): Second run of Varnish benchmark added
Update 3 (Mar 16, 2011): Cherokee benchmark added
Update 4 (Mar 25, 2011): New benchmark with the optimized settings is available

Introduction

Apache is the de facto web server on Unix system. Nginx is nowadays a popular and performant web server for serving static files (i.e. static html pages, CSS files, Javascript files, pictures, …). On the other hand, Varnish Cache is increasingly used to make websites “fly” by caching static content in memory. Recently, I came across a new application server called G-WAN. I’m only interested here in serving static content, even if G-WAN is also able to serve dynamic content, using ANSI C scripting. Finally, I also included Cherokee in the benchmark.

Java NIO really faster than traditional IO ?

According to Jeremy Manson and Paul Tyma, NIO is not faster than the old traditional IO. Paul Tyma argues that since the Native Posix Thread Library (NPTL) arrived in Linux 2.6, multithreading is so cheap that it outperforms the select-based NIO alternative. He goes on to quote some impressive benchmarks from Rahul Bhargava, which show that multithreading gives at least 25% greater throughput that NIO, in a test with 1,700 concurrent connections.

Finally, here are the slides of an interesting presentation given by PaulTyma.

(Large) files transfer with Netcat

From time to time I need to quickly transfer files from an host to another. Setting up an FTP server is painful, giving an SSH account can be dangerous .. so I’ll present here a quick and efficient solution using Netcat:

host1:$ nc -lvp 3333 | tar -xv
host2:$ tar -cv file [files…] | nc -v IP_host1 3333

where host1 is listening on port 3333 for files sent by host2.

Installing a specific package version with APT

In order to install a specific version of a package in Debian, you can edit /etc/apt/preferences or you can simply specify the version directly on the command line.

Let’s see all the choices we have to install clamav-freshclam:

debian-box:/# apt-cache policy clamav-freshclam
clamav-freshclam:
  Installed: 0.90.1dfsg-3etch10
  Candidate: 0.90.1dfsg-3etch10
  Version table:
     0.92.1~dfsg-1volatile1 0
        500 http://apt-cacher etch/volatile/main Packages
 * 0.90.1dfsg-3etch10 0
        800 http://apt-cacher etch/updates/main Packages
        100 /var/lib/dpkg/status
     0.90.1dfsg-3etch9 0
        800 http://apt-cacher etch/main Packages

The default version to be installed is 0.90.1dfsg-3etch10 0, which is the standart Etch version. If we want to install the volatile version:

debian-box:/# apt-get install clamav-freshclam=0.92.1~dfsg-1volatile1
Reading package lists… Done
Building dependency tree… Done
Suggested packages:
  clamav-docs
The following packages will be upgraded:
  clamav-freshclam
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 12.7MB of archives.
After unpacking 3379kB of additional disk space will be used.
Get:1 http://apt-cacher etch/volatile/main clamav-freshclam 0.92.1~dfsg-1volatile1 [12.7MB]
Fetched 12.7MB in 1s (7263kB/s)           
Preconfiguring packages …
[…]

More info here.