Surviving the Slashdot Effect: Bulletproof Load Balancing with HAProxy

There is no sound more terrifying to a sysadmin than the silence of a pager followed immediately by a user screaming that the site is down. You check top and see your load average hitting 50. Apache is spawning child processes until it hits MaxClients, and then it simply stops answering. You aren't just down; you are dead in the water.

If you are running a single LAMP server in 2009, you are playing Russian Roulette with your uptime. Whether it's a Digg frontpage feature or a seasonal sales spike, single-server architectures fail when you need them most.

The traditional enterprise answer is throwing money at the problem: buying a pair of Cisco LocalDirectors or F5 Big-IPs. But for those of us who care about efficiency (and our budget), there is a better way: HAProxy.

Why HAProxy?

HAProxy (High Availability Proxy) is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications. While Nginx 0.7 is making waves as a web server, HAProxy remains the heavyweight champion for pure load balancing logic. It can saturate a Gigabit line with a fraction of the CPU usage required by standard web servers.

Let's look at a battle-tested architecture to decouple your traffic handling from your application logic.

The Architecture

Instead of one massive server, we use a "tier" approach. This requires reliable virtualization—something strictly enforced at CoolVDS using Xen hypervisors—to ensure network I/O isn't stolen by noisy neighbors.

• Load Balancer (LB01): Runs HAProxy. Receives all traffic on port 80.
• Web Node A (WEB01): Apache/PHP.
• Web Node B (WEB02): Apache/PHP.

If WEB01 dies (kernel panic, bad RAM, bad code), HAProxy detects it instantly and routes all traffic to WEB02. Your users notice nothing.

Implementation on CentOS 5

First, grab the package. If it's not in your base repo, check DAG's repository.

Now, let's configure /etc/haproxy/haproxy.cfg. The default config is often cluttered. Here is a clean, production-ready config for version 1.3:

global
    log 127.0.0.1   local0
    maxconn 4096
    user haproxy
    group haproxy
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    retries 3
    option redispatch
    maxconn 2000
    contimeout 5000
    clitimeout 50000
    srvtimeout 50000

listen webfarm 0.0.0.0:80
    mode http
    stats enable
    stats auth admin:secretpassword
    balance roundrobin
    option httpclose
    option forwardfor
    server web01 192.168.1.10:80 check
    server web02 192.168.1.11:80 check

Configuration Breakdown

• balance roundrobin: Distributes requests sequentially. Good for stateless apps.
• option httpclose: Critical. It tells HAProxy to close the connection to the server after the response. Without this, Keep-Alive handling can get messy.
• check: The heartbeat. HAProxy actively probes the backend servers. If one fails, it is removed from rotation.

Pro Tip: If your application relies on PHP sessions stored on the local disk (standard PHP behavior), roundrobin will break your login sessions. You users will log in on Server A, hit reload, land on Server B, and get logged out.

To fix this quickly without rewriting code to use Memcached, use Source IP Hashing by changing the balance mode to: balance source.

The Hardware Reality Check

Software configuration is only half the battle. I have seen perfect HAProxy setups fail because the underlying VPS was choking on I/O wait.

In Norway, data sovereignty is becoming a hot topic under the Personal Data Act (Personopplysningsloven). Hosting outside the country introduces latency and potential compliance headaches with the Data Inspectorate (Datatilsynet).

However, location means nothing if the disk is slow. Most budget VPS providers shove 50 tenants onto a single SATA drive. When one tenant runs a backup, your load balancer stalls.

This is why for mission-critical setups, we stick to CoolVDS. They use 15k RPM SAS drives in RAID-10. It’s the closest you get to dedicated hardware IOPS without the dedicated price tag. Plus, their datacenter in Oslo peers directly with NIX (Norwegian Internet Exchange), dropping latency to sub-5ms for local users.

Testing the Failover

Don't trust it until you break it. With your browser open to the site, go to your terminal and stop Apache on WEB01:

Refresh your browser. If you configured HAProxy correctly, the site loads instantly from WEB02. Check your HAProxy stats page (configured above at /haproxy?stats), and you will see WEB01 marked strictly as DOWN.

Conclusion

High availability isn't just for Google or Yahoo. With tools like HAProxy and a solid virtualization foundation, you can build infrastructure that survives hardware failure and traffic spikes. Stop waking up at 3:00 AM to restart Apache.

Ready to build a cluster that actually stays up? Deploy a high-performance Xen VPS on CoolVDS today and get the I/O throughput your load balancer deserves.