Serving small static files: which server to use ?

Update 1 (Mar 29, 2011): Logging and compression now disabled for all servers
Update 2 (Apr 14, 2011): Added results for Nginx 1.0

Introduction

The goal of this benchmark is to compare several web server or caching server regarding their respective ability and performance to server small static files. The servers are optimized, contrary to my previous benchmark, where each server was tested using its default settings. I consider in this benchmark only the most performant open source servers, namely Varnish Cache, Nginx, Lighttpd, Apache Traffic Server as well as G-WAN (free, but not open source), as it was the clear winner of the previous test.

Load balancing in an Overlay Network

Heterogeneity and Load Balance in Distributed Hash Tables
http://www.cs.berkeley.edu/~pbg/y0.pdf

Summary, key ideas
Instead of picking k virtual servers with random IDs, a node clusters those IDs in a random fraction f(k/n) of the ID space. This allows the node to share a single set of overlay links among all k virtual servers.

Comments
When a node leaves the system (same issue when a node arrives), the total load of the node will be given to its direct neighbor. This can result in cascading errors. A better approach when a node arrives is to pick random IDs and not contiguous IDs, in order to balance the load. A simple way to deal with heterogeneity is that a node hosts an amount of virtual servers proportional to its capacity.
A second issue is that it does not allow movement of virtual servers between nodes.

How to start a PhD in computer science ?

I’m currently trying to write my thesis plan, here are some general advices:

• refine your idea, play with it, try to find several applications of your idea
• don’t try to compete with industrial developments (big players)
• define a clear scope of your idea: which kind of applications will benefit from your idea ?
• define the emphasis on precise points (want to minimize latency, power consumption, latency, … ?)
• define a business model: who is the provider ? Who is the consumer ? What is the service provided ?
• find 3-4 practical or technical problems to solve in order to make your idea concrete
• have a careful look at the current literature, and find out the related works in the field of your idea and in the neighbors fields.

After this process, you should have a small document explaining clearly your novel idea and its direction. Then, it’s recommended to quickly advertise your idea by solving one of the technical problem and publish a paper about it, so that the scientific community is aware of your work.