Stop Letting Default Configs Kill Your API Performance

It is late 2018. We have moved monoliths to microservices, embraced Docker containers, and started orchestrating with Kubernetes 1.12. Yet, I still see developers deploying Nginx as an API gateway using the default apt-get install configuration. This is negligence.

If your users are in Oslo and your servers are in Frankfurt, you are already fighting the speed of light. If your gateway adds another 200ms of SSL handshake overhead and connection churning, your shiny React Native app will feel sluggish. Latency is the new downtime.

I have spent the last month auditing infrastructure for a major Norwegian fintech startup. Their problem wasn't code efficiency; it was the gateway. Here is exactly how we tuned their Nginx layer to handle 10k RPS while keeping the Datatilsynet (Data Protection Authority) happy.

1. The TLS 1.3 Revolution is Here. Use It.

OpenSSL 1.1.1 was released just two months ago (September 2018), and Nginx 1.13+ supports it. If you are not running TLS 1.3, you are wasting round trips. TLS 1.2 requires two round trips for the handshake. TLS 1.3 does it in one (1-RTT).

For a user on a shaky 4G connection in Tromsø connecting to a server in Oslo, that single round trip reduction is noticeable.

Here is the configuration you should be testing in your nginx.conf right now:

ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
ssl_prefer_server_ciphers off; # Let the client choose for TLS 1.3

Note: You need to ensure your underlying OS (like Ubuntu 18.04) has the correct OpenSSL version linked. Run nginx -V to verify.

2. The Upstream Keepalive Trap

This is the most common mistake in microservices. By default, Nginx acts as a courteous HTTP client: it opens a connection to your backend service (Node, Go, Python), sends the request, gets the response, and closes the connection.

If you have high traffic, you are exhausting your ephemeral ports and wasting CPU cycles on TCP handshakes between your gateway and your backend. You need to keep those connections open.

upstream backend_api {
    server 10.0.0.5:8080;
    server 10.0.0.6:8080;
    
    # The magic number. Keep 64 idle connections open to the backend.
    keepalive 64;
}

server {
    location /api/ {
        proxy_pass http://backend_api;
        
        # Essential headers for keepalive to work
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

Pro Tip: Without proxy_set_header Connection "";, Nginx forwards the close header to the backend, defeating the purpose of the keepalive directive. I've seen senior engineers miss this.

3. Kernel Tuning: The Sysctl Layer

Linux defaults are designed for general-purpose usage, not high-throughput API gateways. When you are pushing thousands of requests per second, the kernel's TCP stack becomes a bottleneck.

Edit /etc/sysctl.conf. These settings help deal with the "thundering herd" and rapid connection recycling:

# Allow reusing sockets in TIME_WAIT state for new connections
net.ipv4.tcp_tw_reuse = 1

# Max number of packets in the receive queue
net.core.netdev_max_backlog = 16384

# Increase the maximum number of open file descriptors
fs.file-max = 2097152

# Max connections in the listen queue
net.core.somaxconn = 65535

Apply these with sysctl -p. If you are on a standard VPS, check if your provider allows kernel-level tuning. Many container-based VPS (OpenVZ) won't let you touch this. This is why we insist on KVM virtualization at CoolVDS—you need full kernel control to really tune performance.

4. The Hardware Reality: Why NVMe Matters

You can tune software all day, but if your API Gateway does any buffering, caching, or extensive logging, you are bound by Disk I/O. In 2018, standard SSDs are becoming the baseline, but NVMe (Non-Volatile Memory express) is the differentiator.

When Nginx buffers a large request body to disk (client uploads) or writes to the access.log under heavy load, IOwait spikes. On a SATA SSD, you might see 5-10ms latency spikes. On NVMe, that drops to microseconds.

If your current host is still selling you "High Speed Storage" that turns out to be a shared SATA RAID array, move. The bottleneck is physical.

5. GDPR and Logging Performance

Since May 25th, we are all living in the GDPR era. Logging IP addresses in your API gateway is now a liability. Furthermore, writing every single access line to disk synchronously kills throughput.

We solve both compliance and performance with one buffer strategy:

log_format anonymized '$remote_addr_anon - $remote_user [$time_local] '
                      '"$request" $status $body_bytes_sent '
                      '"$http_referer" "$http_user_agent"';

# Map to anonymize the last octet of the IP
map $remote_addr $remote_addr_anon {
    ~(?P\d+\.\d+\.\d+)\.    $ip.0;
    default                     0.0.0.0;
}

# Buffer logs to memory, write every 5 seconds or when buffer fills
access_log /var/log/nginx/access.log anonymized buffer=32k flush=5s;

This configuration anonymizes the IP (keeping Datatilsynet off your back) and buffers writes to reduce IOPS pressure.

The Norwegian Advantage

Latency is geography. If your customers are in Norway, hosting in London or Amsterdam adds 20-30ms round trip time (RTT) simply due to physics. Hosting in US East adds 90ms+.

By placing your API Gateway on CoolVDS instances in Oslo, you are peering directly at the NIX (Norwegian Internet Exchange). You get low latency, strict Norwegian privacy laws, and green hydroelectric power stability. It is the pragmatic choice for any serious Nordic dev team.

Final Thoughts

Performance isn't an accident; it is architecture. Upgrade to OpenSSL 1.1.1, enable Keepalives, tune your sysctl.conf, and ensure your underlying iron is running NVMe.

Ready to test the difference? Spin up a CoolVDS NVMe instance in our Oslo datacenter. You can be root and editing your Nginx config in less than 55 seconds.