Squeezing Every Millisecond: High-Performance API Gateway Tuning with Nginx

It is 2014, and the patience of the average user has dropped to near zero. With the explosion of mobile apps relying on REST APIs, latency isn't just a metric; it is the difference between a conversion and an uninstall. I recently audited a backend for a major Oslo-based retailer where their API response times were spiking over 800ms during peak hours. The code was fine. The database was indexed. The problem? A default Nginx configuration sitting on top of a cheap, over-sold VPS.

Most hosting providers sell you "RAM" and "Cores," but they hide the Steal Time and I/O Wait. If you are building an API Gateway today, you need raw, predictable throughput. Let’s get into the grime of nginx.conf and sysctl to fix this.

The "Reverse Proxy" Fallacy

Many developers treat Nginx as a simple pass-through. They apt-get install it, leave the defaults, and wonder why Wait times are high. When Nginx acts as an API Gateway, it terminates the connection, handles SSL, and buffers the request before talking to your backend (PHP-FPM, Node.js, or Python).

If you don't tune the connection handling, you are essentially DDoSing yourself.

1. Worker Processes & Connections

The old advice of "one worker per CPU core" stands, but the connection limits in default configs are laughably low for an API handling thousands of concurrent mobile clients.

worker_processes auto;
worker_rlimit_nofile 100000;

events {
    worker_connections 4096;
    use epoll;
    multi_accept on;
}

Critical Detail: The worker_rlimit_nofile directive must be higher than worker_connections. If you hit the file descriptor limit on Linux, Nginx will start dropping requests silently in the error logs. On a CoolVDS KVM instance, we default the underlying OS limits high, but you must explicitly set this in your config.

2. Upstream Keepalive: The Hidden Killer

By default, Nginx opens a new connection to your upstream application (like Node.js listening on port 3000) for every single request. For an API Gateway, this TCP handshake overhead is catastrophic.

You must enable keepalive connections to the backend.

upstream backend_api {
    server 127.0.0.1:3000;
    # Keep 64 idle connections open to the backend
    keepalive 64;
}

server {
    location /api/ {
        proxy_pass http://backend_api;
        # Required for keepalive to work
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

Pro Tip: Without clearing the Connection header, Nginx forwards the client's "close" header to the backend, killing the keepalive tunnel you just tried to create. I've seen this mistake cost 30ms per request.

3. TCP Stack Tuning (sysctl.conf)

Nginx can only be as fast as the Linux kernel allows. In 2014, the defaults on Ubuntu 12.04 or CentOS 6 are conservative, designed for the dial-up era. We need to allow rapid recycling of TCP sockets, especially for handling the TIME_WAIT state common in high-throughput API environments.

Add this to /etc/sysctl.conf and run sysctl -p:

# Allow reuse of sockets in TIME_WAIT state for new connections
net.ipv4.tcp_tw_reuse = 1

# Increase the maximum number of open files
fs.file-max = 2097152

# Max backlog for connection requests (protects against SYN floods)
net.core.somaxconn = 65535

# Increase TCP buffer sizes for modern bandwidth
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

4. The SSL/TLS Overhead (Heartbleed Note)

Given the massive Heartbleed (CVE-2014-0160) vulnerability disclosed last week, I assume you have already updated OpenSSL to 1.0.1g. If not, stop reading and patch your servers immediately.

Security aside, SSL handshakes are heavy. For an API, you want to enable session caching so clients don't renegotiate SSL on every call. Also, enforce the use of strong ciphers to avoid retry compatibility modes.

ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;

# Prioritize server ciphers to enforce performance
ssl_prefer_server_ciphers on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

Infrastructure Matters: The Noisy Neighbor Problem

You can have the most optimized Nginx config in the world, but if your database is waiting on disk I/O, your API is slow. This is the dirty secret of "Cloud" hosting. Most providers use shared storage (SAN). When another customer on the same node decides to run a backup, your API latency spikes.

Comparison: Container vs. KVM

At CoolVDS, we strictly use KVM virtualization. This means your memory is yours. It is not burstable; it is reserved. We use local SSD arrays (RAID 10) rather than network storage. For a Norwegian business, this also means your data physically resides in our Oslo data center, ensuring low latency to the NIX (Norwegian Internet Exchange) and compliance with the Personal Data Act (Personopplysningsloven).

Conclusion

Performance isn't an accident. It is a result of peeling back the layers of abstraction and tuning the engine. By optimizing Nginx workers, enabling upstream keepalives, and tweaking the Linux kernel, you can handle 10x the traffic on the same hardware.

However, software tuning hits a wall if the hardware lies to you. Don't let disk I/O kill your application's responsiveness. Deploy a test instance on a CoolVDS KVM slice today and see what true isolation feels like.