Surviving the Digg Effect: Bulletproof Load Balancing with HAProxy 1.3

It’s 3:00 AM. Your pager goes off. Nagios is screaming because your primary Apache server just hit a load average of 50.0 and swapped itself to death. Why? Because you landed on the front page of Digg, and your single-server architecture just folded like a cheap lawn chair.

If you are running a serious business in Norway, relying on a single box is not a strategy; it’s a liability. We see this constantly at CoolVDS: developers deploying heavy Magento or Drupal sites on a single slice, hoping the hardware gods will be kind. They rarely are.

Hardware load balancers like F5 Big-IP are fantastic if you have the budget of a bank. For the rest of us, there is HAProxy. It is free, open-source, and frankly, more flexible than most proprietary iron.

The Architecture of Availability

The concept is simple. You place a lightweight HAProxy instance in front of your web servers (backends). It accepts the traffic and distributes it based on availability and load. If Web Node A dies, HAProxy detects it instantly and routes traffic to Web Node B. No downtime. No angry calls from the CEO.

Here is a battle-tested configuration we use for high-traffic clients. This assumes you are running HAProxy 1.3 on a stable Debian Lenny or CentOS 5 system.

global
    log 127.0.0.1   local0
    maxconn 4096
    user haproxy
    group haproxy
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    retries 3
    option  redispatch
    maxconn 2000
    contimeout 5000
    clitimeout 50000
    srvtimeout 50000

listen  webfarm 0.0.0.0:80
    mode http
    stats enable
    stats auth admin:password
    balance roundrobin
    cookie JSESSIONID prefix
    option httpclose
    option forwardfor
    server web01 10.0.0.1:80 cookie A check
    server web02 10.0.0.2:80 cookie B check

Breaking Down the Config

The balance roundrobin directive is crucial here. It rotates requests evenly. However, if you are running a PHP application with local sessions, users will get logged out if they bounce between servers. That is where cookie JSESSIONID prefix comes in. It injects a cookie to ensure session stickiness.

Also, notice option forwardfor. Without this, your backend Apache logs will only show the IP address of the load balancer, not the actual customer. This header passes the real client IP through, which is essential for audit logs required by Datatilsynet here in Norway.

Latency Matters: The NIX Factor

In the Nordic region, latency is the silent killer of conversion rates. If your servers are hosted in a budget datacenter in Texas, your Norwegian users are dealing with 150ms+ ping times. That delay makes TCP handshakes sluggish.

At CoolVDS, our infrastructure is peered directly at NIX (Norwegian Internet Exchange) in Oslo. When you put a load balancer on our network, the round-trip time to local ISPs like Telenor or NextGenTel is often under 5 milliseconds. Speed is a feature.

SysAdmin Tip: Avoid the "noisy neighbor" problem common with cheap VPS hosting. Many providers use OpenVZ and oversell RAM. If another user on the node gets hit by a DDoS, your database locks up. We use Xen HVM virtualization with strict resource isolation. You get the RAM you pay for.

Storage IO: The Bottleneck

Load balancing solves CPU limitations, but it does not fix bad I/O. If your MySQL database is struggling to write to disk, adding more web servers won't help. You need fast storage.

While Solid State Drives (SSDs) like the Intel X25-E are starting to enter the enterprise market, they are still cost-prohibitive for mass storage. The pragmatic solution for 2010 is RAID-10 with SAS 15k RPM drives. This gives you the redundancy of mirroring with the speed of striping. It is the standard on all CoolVDS performance plans.

Security and Compliance

Under the Personal Data Act (Personopplysningsloven), you are responsible for the integrity of your user data. HAProxy helps here too. By offloading SSL termination (though Stunnel is often needed in front of HAProxy 1.3 for HTTPS), you create a secure gateway. You can also block malicious IPs at the balancer level before they ever hit your application code.

Final Thoughts

Scaling isn't just about adding more servers; it's about control. HAProxy gives you that control. It turns a fragile single-point-of-failure into a robust cluster.

Don't wait for your site to crash during the next traffic spike. Deploy a redundant cluster on CoolVDS today. With our 15k SAS storage and NIX peering, your infrastructure will be ready for anything the internet throws at it.