Scaling Past C10k: Bulletproof Nginx Reverse Proxy Configuration on CentOS 6

It is 3:00 AM. Your load average is hitting 50.00. The kernel is killing processes to save itself (OOM Killer), and your client in Oslo is screaming because their e-commerce store is timing out during a flash sale. If you are still serving static assets directly through Apache prefork, you deserve the pager duty alert waking you up.

The reality of hosting in 2012 is that hardware is getting cheaper, but concurrency is skyrocketing. The old LAMP stack model, where Apache handles everything from PHP execution to serving favicon.ico, is dead for high-traffic sites. Apache processes are heavy; they consume megabytes of RAM just to wait for a slow client on a 3G connection.

The solution isn't "buy more RAM." The solution is architecture. Specifically, placing Nginx in front of your backend servers.

The Architecture: Event-Driven vs. Process-Based

In a standard scenario, I see sysadmins running Apache with MaxClients set to 256. Once you hit 257 concurrent connections, the 258th user waits. If you raise that limit, you swap to disk, and the server grinds to a halt. This is the classic C10k problem.

Nginx uses an asynchronous, event-driven architecture. It doesn't spawn a new process for every connection. It handles thousands of connections in a single worker process with a tiny memory footprint. By placing Nginx on port 80 and Apache on port 8080 (or binding to localhost), Nginx buffers the slow client connections, and only talks to Apache when it needs dynamic content generated. Apache does the heavy lifting, hands the result back to Nginx instantly, and frees up the worker.

Pro Tip: Never rely on OpenVZ containers for this type of high-load architecture. You need dedicated kernel resources for network stack tuning. At CoolVDS, we strictly use KVM virtualization to ensure your sysctl.conf changes actually take effect and aren't overridden by a host node policy.

Step 1: Installation and Basic Setup

We are assuming a clean CentOS 6.2 install. First, we need the EPEL repository, as Nginx isn't in the base CentOS repos.

rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-7.noarch.rpm
yum install nginx
chkconfig nginx on

Verify you have at least Nginx version 1.0.15 or the newer stable 1.2.0. The 1.2 branch brings significant improvements in upstream keepalive connections.

Step 2: Optimizing nginx.conf for Hardware

Out of the box, Nginx is conservative. We need to tune it for the hardware available on your VPS. If you are running on our high-performance SSD plans, you can push the buffer sizes, but watch your RAM usage.

Edit /etc/nginx/nginx.conf:

user nginx;
worker_processes auto; # Automatically detects CPU cores
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 2048; # Increase this if you expect heavy traffic
    use epoll; # Essential for Linux 2.6+ kernels
    multi_accept on;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    
    # Optimization for file serving
    sendfile        on;
    tcp_nopush      on;
    tcp_nodelay     on;
    
    # Timeouts to prevent slowloris attacks
    keepalive_timeout  65;
    client_body_timeout 10;
    client_header_timeout 10;
    send_timeout 10;

    # Hide version to annoy script kiddies
    server_tokens off;

    include /etc/nginx/conf.d/*.conf;
}

Step 3: The Proxy Configuration

This is where the magic happens. We need to tell Nginx how to forward traffic to the backend (Apache running on 127.0.0.1:8080). Create a file called /etc/nginx/conf.d/proxy_params so we can reuse it across vhosts.

proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;

# Buffer settings - Critical for high throughput
proxy_buffers 8 16k;
proxy_buffer_size 32k;

Now, define your virtual host in /etc/nginx/conf.d/default.conf:

server {
    listen       80;
    server_name  example.no www.example.no;

    # Serve static files directly - Bypassing Apache entirely
    location ~* \.(jpg|jpeg|gif|png|css|js|ico|xml)$ {
        root /var/www/html;
        expires 30d;
        access_log off;
    }

    # Pass everything else to Apache
    location / {
        proxy_pass http://127.0.0.1:8080;
        include /etc/nginx/conf.d/proxy_params;
    }
}

The