Squeezing Every Millisecond: API Gateway Tuning for the Nordic Market

Your API is slow. It’s not your code. It’s not your database. It’s the gatekeeper.

In a distributed architecture, the API Gateway is the single most critical point of failure. I recently audited a high-traffic fintech platform based in Oslo. They were throwing money at larger instances, yet their 99th percentile latency (p99) during peak trading hours was erratic. The culprit wasn't CPU exhaustion. It was a default Linux network stack colliding with an untuned Nginx configuration.

If you are serving the Norwegian market, latency isn't just about speed; it's about trust. A request taking 40ms to round-trip to Frankfurt might seem acceptable, but when you chain five microservices, that overhead compounds. Hosting locally in Norway on CoolVDS lowers that baseline physical latency to single digits for your local users. But hardware is only half the battle. You need to tune the engine.

The "Open Files" Lie

Most tutorials tell you to just increase ulimit -n. That is amateur hour. In December 2021, on a modern kernel (like the 5.4 or 5.10 kernels we run on our KVM nodes), you need to align the file descriptors with the worker connections.

If you set Nginx worker_connections to 1024 (the default) but your API Gateway handles thousands of concurrent keep-alive connections to backend microservices, you will hit silent drops. Not errors. Drops.

First, check your current limits:

ulimit -n
# If this returns 1024, you are throttled.

Here is the production-ready configuration for /etc/security/limits.conf to handle a "thundering herd" scenario:

* soft nofile 100000
* hard nofile 100000
root soft nofile 100000
root hard nofile 100000

Kernel Tuning via Sysctl

The Linux defaults are designed for general-purpose computing, not high-throughput packet switching. We need to modify /etc/sysctl.conf to allow the OS to recycle connections faster. This is critical for preventing port exhaustion when communicating with backend services.

# Maximize the backlog of incoming connections
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535

# Allow reusing sockets in TIME_WAIT state for new connections
net.ipv4.tcp_tw_reuse = 1

# Increase local port range to avoid exhaustion
net.ipv4.ip_local_port_range = 1024 65535

# TCP Fast Open (TFO) reduces handshake round trips
net.ipv4.tcp_fastopen = 3

Apply these with sysctl -p. If you are running on a budget VPS provider that uses container virtualization (like OpenVZ), you often cannot modify these kernel parameters. This is why CoolVDS strictly uses KVM virtualization. You get your own kernel. You get full control.

Nginx: The Upstream Keepalive Trap

This is where 90% of setups fail. By default, Nginx acts as a reverse proxy that opens a new connection to your backend (Node.js, Go, Python) for every single request. This adds the overhead of a TCP handshake to every API call. It destroys performance.

You must configure the upstream block to keep connections open.

upstream backend_api {
    server 10.0.0.5:8080;
    server 10.0.0.6:8080;

    # The secret sauce: Keep 64 idle connections open to the backend
    keepalive 64;
}

And crucially, you must enforce HTTP 1.1 in the proxy pass, because HTTP 1.0 (the default for proxying) does not support keepalive:

server {
    listen 80;
    listen 443 ssl http2;
    
    location /api/ {
        proxy_pass http://backend_api;
        
        # REQUIRED for keepalive to work
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        
        # Buffer tuning for high throughput
        proxy_buffers 16 16k;
        proxy_buffer_size 32k;
    }
}

Pro Tip: Monitor your "TIME_WAIT" sockets. If you see thousands of them stacking up using `ss -s`, your keepalive settings are failing. Efficient systems reuse sockets.

The Hardware Reality: NVMe & Local Latency

Software tuning hits a wall if the underlying disk I/O is slow. API Gateways log heavily. Access logs, error logs, and local caching rely on disk write speeds. On a standard HDD or even a cheap SATA SSD, high-concurrency logging can block the Nginx worker process.

We benchmarked disk I/O impact on request latency. The results were stark.

Furthermore, geography dictates physics. If your users are in Oslo or Bergen, hosting in a German or US datacenter adds 20-100ms of unavoidable network latency. With the Schrems II ruling invalidating the Privacy Shield, keeping data within Norwegian or purely EEA borders is no longer just a performance tactic—it is a compliance necessity for handling Norwegian user data.

Conclusion: Don't settle for "Default"

An API Gateway is a precision instrument. Leaving it on default settings is like buying a Ferrari and driving it in first gear. You need a dedicated kernel, fast NVMe storage for logs, and a location close to your users.

Stop fighting noisy neighbors and stolen CPU cycles. Deploy your optimized gateway on a platform built for performance.

Ready to drop your latency? Spin up a high-performance NVMe KVM instance on CoolVDS in Oslo today.