Cutting 50ms Off Your API Response Time: Gateway Tuning for Northern Europe

If your API Gateway takes 200ms to process a request before it even hits your microservices, you have already lost. In the Nordic market, where user expectations for digital services are arguably the highest in Europe, latency is the silent killer of user retention.

I recently audited a fintech setup in Oslo. They were baffled. Their Kubernetes clusters were idling, their databases were optimized, yet their mobile app felt sluggish. The culprit? A default configuration NGINX ingress controller struggling to handle TLS handshakes and TCP connection churn. They were treating their Gateway as a passthrough, not a high-performance compute node.

Let's fix that. We are going to look at the Linux kernel, SSL termination, and the underlying hardware constraints that exist in 2022. No fluff, just engineering.

1. The Linux Kernel: Tuning the Network Stack

Most Linux distributions ship with generic settings designed for file servers or desktops, not high-throughput API Gateways handling thousands of ephemeral connections. If you are running your gateway on a standard image without touching sysctl.conf, you are throttling yourself.

The first bottleneck you will hit is the ephemeral port exhaustion and the TIME_WAIT state. When your API Gateway talks to upstream services (backend APIs), it opens a socket. When that closes, it sits in TIME_WAIT. High traffic means you run out of sockets.

Apply these settings to your /etc/sysctl.conf on your CoolVDS instance:

# Allow reuse of sockets in TIME_WAIT state for new connections
net.ipv4.tcp_tw_reuse = 1

# Increase the range of ephemeral ports
net.ipv4.ip_local_port_range = 1024 65535

# Max number of packets in the receive queue
net.core.netdev_max_backlog = 16384

# Increase the maximum number of open files (file descriptors)
fs.file-max = 500000

# TCP Fast Open (TFO) to reduce handshake RTT
net.ipv4.tcp_fastopen = 3

Why this matters: tcp_fastopen allows data to be sent during the SYN packet of the TCP handshake. For a user connecting from Tromsø to a server in Oslo, this saves a full round-trip time. In a high-latency mobile environment, that is perceptible speed.

2. NGINX & Kong: The Upstream Keepalive Mistake

Whether you use raw NGINX, Kong, or OpenResty, a common error is failing to enable keepalives to the upstream services. By default, NGINX acts as a polite HTTP/1.0 client to your backend: it opens a connection, sends the request, gets the response, and closes the connection.

For SSL-encrypted backends, this forces a full TLS handshake for every single API call between your gateway and your microservice. That is CPU suicide.

Here is the correct configuration pattern:

upstream backend_service {
    server 10.0.0.5:8080;
    
    # The critical directive. Keeps 32 idle connections open.
    keepalive 32;
}

server {
    location /api/ {
        proxy_pass http://backend_service;
        
        # Required to enable keepalive
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

Pro Tip: If you are using Kong (v2.x), check your `upstream_keepalive_pool_size` property. Defaults are often too low for high-throughput production environments. Bump it up to at least 60 for busy services.

3. TLS Termination: The Nordic Latency Tax

Norway is long. The physics of light through fiber optics dictates that a handshake between Kirkenes and Oslo takes time. If your certificate chain is heavy or you aren't using OCSP Stapling, you are adding 100ms+ of "dead time" before a byte of application data is sent.

Ensure your SSL configuration favors speed without sacrificing security. In early 2022, TLS 1.3 is mandatory for performance.

ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256;
ssl_prefer_server_ciphers on;

# Enable OCSP Stapling so the client doesn't have to query the CA
ssl_stapling on;
ssl_stapling_verify on;
resolver 1.1.1.1 8.8.8.8 valid=300s;
resolver_timeout 5s;

We use ECDSA certificates on our internal CoolVDS load balancers. They offer the same security level as RSA but with much smaller keys, resulting in faster handshakes. If your CA supports it, switch to ECDSA.

4. The Hardware Reality: Steal Time and Noisy Neighbors

You can tune your kernel all day, but if your virtualization platform is stealing CPU cycles, it's pointless. In a "Public Cloud" environment, your API Gateway might be sharing a physical core with a neighbor running a crypto miner or a heavy batch job. This manifests as "CPU Steal" time.

For an API Gateway, consistency > raw burst speed. You need predictable interrupt handling.

At CoolVDS, we enforce strict KVM isolation. When you provision a 4-core instance, those cycles are reserved. For an API Gateway processing 10,000 req/sec, this elimination of "noisy neighbor" jitter stabilizes the p99 latency significantly.

5. Benchmarking: Don't Guess, Verify

Do not rely on browser dev tools for testing API performance. You need concurrency. I use wrk (available in most 2022 repos) to stress test the configuration.

Run this from a separate machine (preferably close to your target geo, e.g., if your server is in Oslo, test from Stockholm or local):

# Run for 30 seconds, using 12 threads, keeping 400 open connections
wrk -t12 -c400 -d30s --latency https://api.yourdomain.no/health

Look specifically at the Stdev (Standard Deviation) and the Max latency. A low average means nothing if 5% of your users (the p95) are waiting 3 seconds.

Compliance and Data Sovereignty

Performance isn't just about speed; it's about legality. Post-Schrems II, routing traffic through US-owned gateways can be a compliance nightmare for Norwegian companies. Hosting your gateway on CoolVDS ensures data stays within the EEA, on infrastructure not subject to the US CLOUD Act nuances that plague the hyperscalers. The Datatilsynet (Norwegian Data Protection Authority) is watching closely; architecture is your first line of defense.

Conclusion

API Gateway performance is an accumulation of marginal gains. It is 5ms from the kernel TCP stack, 15ms from TLS 1.3, 20ms from upstream keepalives, and the crucial stability of dedicated hardware.

Don't let slow I/O or shared CPU cycles kill your SEO or user experience. You need raw, predictable compute.

Ready to lower your latency? Deploy a high-frequency NVMe instance on CoolVDS today and test the difference yourself.