Apache vs Lighttpd: Surviving the C10k Problem on Norwegian Infrastructure

If you are still thinking that throwing more RAM at a sluggish web server is the solution, you are doing it wrong. In the world of high-traffic hosting, the architecture of your daemon matters more than the raw gigahertz of your CPU. We are seeing a shift in the hosting landscape in late 2009. The "C10k problem"—handling ten thousand concurrent connections—is no longer just a theoretical paper by Dan Kegel; it is a reality for any growing media site or social platform in Scandinavia.

For years, the LAMP stack (Linux, Apache, MySQL, PHP) has been the de-facto standard. But as traffic spikes hit, Apache’s process-based model is showing its age. Enter Lighttpd (pronounced "lighty"), the event-driven challenger that promises to serve static content faster than you can blink, with a memory footprint that barely registers. Today, we aren't just reading man pages; we are benchmarking these two beasts on a standard CentOS 5.4 build to see which one deserves the public IP on your server.

The Architecture War: Threads vs. Events

The fundamental difference lies in how they handle connections. Apache 2.2, by default on most distributions, runs the prefork MPM (Multi-Processing Module). This is safe, stable, and compatible with non-thread-safe libraries (looking at you, older PHP extensions). However, it scales linearly with memory. Every incoming connection spawns a dedicated process or grabs one from a pool. If you have 500 concurrent users, you have 500 Apache processes eating RAM.

Lighttpd takes a different approach. It uses an asynchronous, event-based architecture (using epoll on Linux 2.6 kernels). It handles thousands of connections in a single process loop. It doesn't block. It doesn't spawn a heavy child process just to serve a 2KB CSS file. This efficiency is critical when you are running on a Virtual Private Server (VPS) where resources are finite.

Scenario: The "Melting Point"

Last week, we migrated a high-traffic forum based in Oslo from a shared hosting environment to a dedicated CoolVDS KVM slice. They were running Apache 2.2 with default settings. At 6:00 PM, when Norwegian users got home and hit the site, the server Load Average spiked to 25.0. The RAM swapped out to disk, and the site died. The culprit? MaxClients was set too high for the available RAM, causing thrashing.

Pro Tip: Never trust the default httpd.conf shipping with RHEL/CentOS. It is optimized for compatibility, not survival. Always calculate MaxClients = (Total RAM - OS RAM - DB RAM) / Avg Apache Process Size.

Configuration Face-Off

Let's look at the complexity of tuning these servers. Apache is notorious for its XML-like configuration sprawl.

Apache 2.2 Optimization (httpd.conf)

To make Apache survive a traffic spike, you must switch to the Worker MPM if your modules allow it, or strictly tune the Prefork MPM. Here is a battle-tested configuration for a server with 2GB RAM:

<IfModule mpm_prefork_module>
    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10
    # Do not set this higher than your RAM can hold!
    MaxClients          150
    MaxRequestsPerChild 1000
</IfModule>

# Turn off KeepAlive if you are serving dynamic content to many users
# to free up slots faster, or lower the timeout drastically.
KeepAlive On
KeepAliveTimeout 2

Note the MaxRequestsPerChild. We set this to 1000 to kill off processes periodically, mitigating memory leaks in PHP extensions—a common headache in long-running Apache instances.

Lighttpd Simplicity (lighttpd.conf)

Lighttpd is refreshing. The configuration looks more like code. It is logical. Here is how we set up a high-performance static server that offloads PHP via FastCGI:

server.modules = (
  "mod_access",
  "mod_fastcgi",
  "mod_accesslog",
  "mod_compress"
)

server.max-fds = 2048
server.event-handler = "linux-sysepoll"
server.network-backend = "linux-sendfile"

# Compression saves bandwidth, vital for Norwegian ADSL users
compress.cache-dir = "/var/cache/lighttpd/compress/"
compress.filetype  = ("text/plain", "text/html", "text/css", "text/javascript")

fastcgi.server = ( ".php" =>
  (( "socket" => "/tmp/php-fastcgi.socket",
     "bin-path" => "/usr/bin/php-cgi",
     "min-procs" => 1,
     "max-procs" => 5,
     "max-load-per-proc" => 4,
     "idle-timeout" => 20
  ))
)

By using linux-sendfile, Lighttpd instructs the kernel to copy data directly from the disk cache to the network card buffer. The CPU barely touches the data. This is why Lighttpd can saturate a 100Mbit line while Apache is still context switching.

Benchmark: Static File Performance

We ran ab (Apache Bench) against both servers hosted on a CoolVDS instance with 512MB RAM and 15k RPM SAS storage. The target: a 10KB static image file, requested 10,000 times with 100 concurrency.

The numbers don't lie. For static content—images, CSS, JS—Lighttpd obliterates Apache. It uses a fraction of the RAM and serves double the requests. If you are hosting a heavy media site, putting Lighttpd in front of your stack is a no-brainer.

The PHP Complication

Here is where the "Pragmatic CTO" pauses. Apache makes PHP easy. You install mod_php, restart httpd, and it works. The PHP interpreter is embedded inside the Apache process. It is compatible with everything, including .htaccess files which many CMSs like WordPress rely on heavily.

Lighttpd does not support .htaccess. You must translate those rewrite rules into lighttpd.conf syntax. Furthermore, Lighttpd talks to PHP via FastCGI. In 2009, managing FastCGI processes can be tricky. If the PHP process dies, Lighttpd returns a 500 error. You often need a separate process manager like spawn-fcgi to keep things stable.

However, separating the web server from the PHP processor is cleaner architecture. It allows you to distribute the load. You could have one CoolVDS instance handling the Lighttpd frontend and three internal instances processing the PHP logic.

Hardware Reality: Disk I/O Latency

Regardless of which web server you choose, your bottleneck in 2009 is likely disk I/O. Apache logging is synchronous. If your disk array is slow, Apache waits. This is why choosing the right hosting partner is critical. Many budget VPS providers in Europe oversell their storage arrays. You might get a CPU, but your iowait will skyrocket.

At CoolVDS, we utilize Enterprise SAS 15,000 RPM drives in RAID-10 configurations. We don't rely on standard SATA drives for production workloads. When we define a KVM virtual machine, we allocate dedicated I/O throughput. This low latency is essential when your database (MySQL) and your web server are fighting for disk access during peak hours.

The Norwegian Data Context

Operating in Norway adds another layer: compliance. Under the Personal Data Act (Personopplysningsloven), you are responsible for the security and integrity of user data. Stability is part of security. A crashed server is a vulnerability. Hosting locally in Oslo not only reduces latency to the NIX (Norwegian Internet Exchange) to under 5ms, but it also ensures you aren't routing sensitive data across borders unnecessarily, keeping the Datatilsynet happy.

Verdict: Which One to Choose?

Choose Apache 2.2 if:

• You need broad compatibility with .htaccess files.
• You are running complex legacy apps that rely on specific Apache modules.
• Your team is small and prefers "it just works" over raw performance.

Choose Lighttpd 1.4 if:

• You are serving high-volume static content (images, downloads).
• You are comfortable with FastCGI and rewriting config files.
• You need to squeeze maximum performance out of a limited RAM budget.

Ultimately, the software is only as good as the infrastructure it runs on. You can tune Lighttpd to perfection, but if your host's network is congested or the virtualization is oversold, you will still time out. Don't let slow I/O or "noisy neighbors" kill your SEO rankings.

Ready to test the difference? Deploy a CoolVDS instance with Enterprise SAS storage today and see how your Load Average drops. Your users (and your sysadmin) will thank you.