Optimizing NGINX as an API Gateway: Survival Guide for High-Traffic Nodes

It is almost 2016. If you are still running a monolithic LAMP stack for a mobile backend, you are already behind. The industry is shifting aggressively toward microservices. We break the monolith, we decouple the logic, and we containerize with Docker.

But there is a hidden cost.

Latency. When function calls become HTTP requests, milliseconds matter. A single API call from a client might fan out to five internal services. If your API Gateway—the traffic cop sitting at the edge—adds even 50ms of overhead, your user experience degrades immediately. I have seen perfectly good code fail because the infrastructure layer choked on TCP handshakes.

This guide is for the sysadmins and DevOps engineers who are tired of seeing "Connection Timed Out" in their logs. We are going to tune the Linux kernel and NGINX to handle the thundering herd.

1. The OS Layer: Linux Default Settings Are Wrong

Most Linux distributions, including the minimal CentOS 7 or Ubuntu 14.04 images you likely use, ship with conservative network defaults. They assume you are running a file server, not a high-concurrency API gateway.

When you act as a gateway, you chew through ephemeral ports. If you don't tune sysctl, you will hit the wall where the kernel cannot recycle TCP connections fast enough. You'll see TIME_WAIT spikes in netstat.

Here is the baseline configuration I deploy on every CoolVDS instance acting as an edge node. Edit /etc/sysctl.conf:

# Increase system-wide file descriptors
fs.file-max = 2097152

# Widen the port range to allow more concurrent connections
net.ipv4.ip_local_port_range = 1024 65535

# Reuse sockets in TIME_WAIT state for new connections (safe for internal backend comms)
net.ipv4.tcp_tw_reuse = 1

# Increase the backlog for incoming connections
net.core.somaxconn = 32768
net.ipv4.tcp_max_syn_backlog = 8192

# Disconnect dead TCP connections faster (default is usually too long)
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_fin_timeout = 30

Apply these changes with sysctl -p. Without this, no amount of NGINX tweaking will save you.

Pro Tip: On a virtualized environment, check your conntrack table limits. If you are on a cheap VPS using OpenVZ, you might hit the host's barrier. This is why we exclusively use KVM at CoolVDS—you get your own kernel and your own limits.

2. NGINX Configuration: Beyond the Basics

NGINX is efficient, but the default nginx.conf is often set for compatibility, not speed. For an API Gateway, we care about two things: keeping connections open to the backend (to avoid handshake overhead) and handling high concurrency from the client.

Worker Processes and Limits

Set worker_processes to auto. But more importantly, check worker_rlimit_nofile. This directive controls the maximum file descriptors (FDs) a worker can open. If this is lower than your OS limit, NGINX is the bottleneck.

worker_processes auto;
worker_rlimit_nofile 65535;

events {
    worker_connections 16384;
    use epoll;
    multi_accept on;
}

Upstream Keepalive

This is the most common mistake I see in audits. By default, NGINX talks to backends using HTTP/1.0 and closes the connection after every request. If your gateway proxies to a Node.js or Go service, you are wasting CPU cycles tearing down and rebuilding TCP sockets.

Enable keepalive in your upstream block:

upstream backend_api {
    server 10.0.0.5:8080;
    server 10.0.0.6:8080;
    
    # Keep 64 idle connections open to the backend
    keepalive 64;
}

server {
    location /api/ {
        proxy_pass http://backend_api;
        
        # Required for keepalive to work
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        
        # Pass the real IP to your app
        proxy_set_header X-Real-IP $remote_addr;
    }
}

3. The SSL Tax: Security vs. Speed

With Google now using HTTPS as a ranking signal and the recent release of HTTP/2 (supported in NGINX 1.9.5+), encryption is mandatory. But RSA handshakes are heavy.

We need to use Elliptic Curve Cryptography (ECC). It provides equivalent security to RSA with much smaller key sizes, meaning less CPU usage during the handshake. This is critical for mobile clients on 3G/4G networks in rural Norway.

Ensure you are prioritizing ECDHE cipher suites. Here is a 2015-compliant config that scores an A+ on SSL Labs:

ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK';
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;

4. The "Steal Time" Problem

You can have the best NGINX config in the world, but if your underlying host is oversold, your metrics will fluctuate wildly. In virtualized environments, "CPU Steal" is the percentage of time your virtual CPU waits for the physical CPU to attend to it.

If you are seeing random latency spikes during peak hours (like 8:00 PM CET when everyone starts streaming Netflix), your neighbor is likely noisy. This is why strict resource isolation matters.

Comparison: Standard VPS vs. Dedicated Resources

We built CoolVDS on KVM because we need guarantees. When we say you get 2 vCPUs, those cycles are reserved for you.

5. Data Sovereignty and The "Schrems I" Fallout

We cannot ignore the legal landscape. In October 2015, the European Court of Justice invalidated the Safe Harbor agreement (Schrems I). If you are a Norwegian business storing customer data, relying blindly on US-based cloud giants is now a legal minefield.

Datatilsynet (The Norwegian Data Protection Authority) is becoming stricter. Hosting your API Gateway and databases within Norway or the EEA isn't just about latency—though 3ms pings to Oslo are nice—it is about compliance. Keeping traffic local via NIX (Norwegian Internet Exchange) ensures your data stays within a legal jurisdiction you understand.

Summary

Performance is a stack, not a switch. It starts with the hardware (fast I/O), moves to the kernel (TCP tuning), and ends with the application config (NGINX). Neglect one, and the others are useless.

If you are ready to test this configuration on hardware that doesn't steal your CPU cycles, spin up a CoolVDS instance. We provide the raw power; you provide the code.