Squeezing Milliseconds: High-Performance API Gateway Tuning for Nordic Workloads

If you think a standard Nginx install or a default HAProxy configuration is enough for a production API gateway, you are already losing money. I see it every week. A startup in Oslo calls me because their mobile app feels "sluggish." They blame the 4G network in Finnmark. They blame the React Native code. But 9 times out of 10, I look at their gateway and see a bottleneck that shouldn't exist.

Your API Gateway is the bouncer of your infrastructure. If the bouncer is slow, nobody gets into the club. In the context of 2019's microservices explosion, where a single user action might trigger five internal service calls, adding 20ms of latency at the gateway compounds into 100ms of delay for the end user. That is unacceptable.

This is not a guide on how to install Kong or Tyk. This is a guide on how to stop them from choking under load. We are going to look at Linux kernel parameters, file descriptors, and why the hardware underlying your VPS—specifically the virtualization technology—matters more than your code.

The Hardware Reality: Steal Time is the Enemy

Before we touch a single config file, we need to address the environment. In 2019, cloud computing is ubiquitous, but "noisy neighbors" remain the silent killer of API performance. If you are running your gateway on a budget VPS where the host CPU is oversubscribed by 400%, your precise kernel tuning means nothing.

You need to check for CPU Steal Time. This metric tells you how long your virtual CPU waits for the physical hypervisor to give it attention.

$ top
# Look at the %st value in the CPU row.
Cpu(s):  1.2%us,  0.5%sy,  0.0%ni, 98.1%id,  0.1%wa,  0.0%hi,  0.0%si,  0.1%st

If %st is consistently above 1-2%, move your workload. For API gateways, which require rapid context switching and interrupt handling, you need dedicated resources. This is why at CoolVDS, we enforce strict KVM isolation. We don't oversell cores. When you request an API response, the CPU instructions execute immediately, not after your neighbor's Bitcoin miner yields the processor.

The Kernel Layer: Breaking the Limits

Linux defaults are designed for general-purpose usage, not for handling 10,000 concurrent connections per second. If you deploy a default Ubuntu 18.04 LTS instance, you will hit the wall on ephemeral ports and open files.

I recently audited a Magento cluster where the gateway was dropping connections during flash sales. The logs showed nothing useful, but dmesg hinted at table overflows. Here is the exact sysctl configuration we applied to fix it.

1. Open File Descriptors

Everything in Linux is a file. A socket is a file. Nginx needs a file descriptor for every incoming connection and every upstream connection. The default limit is often 1024. Ridiculously low.

# /etc/security/limits.conf
*       soft    nofile  65535
*       hard    nofile  65535
root    soft    nofile  65535
root    hard    nofile  65535

2. TCP Stack Tuning

Edit /etc/sysctl.conf. These settings optimize how the kernel handles the TCP state machine, specifically targeting the TIME_WAIT state which plagues high-throughput gateways.

# Allow reuse of sockets in TIME_WAIT state for new connections
net.ipv4.tcp_tw_reuse = 1

# Increase the range of ephemeral ports (outgoing connections to upstream)
net.ipv4.ip_local_port_range = 1024 65000

# Max backlog of connection requests (vital for burst traffic)
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 8192

# Fast Open can reduce latency by one RTT (if supported by client/server)
net.ipv4.tcp_fastopen = 3

Apply these with sysctl -p. Do not blindly copy-paste settings you find on Stack Overflow from 2012. The tcp_tw_recycle option, for example, is dangerous in NAT environments and was removed in newer kernels, but tcp_tw_reuse is safe and necessary.

Nginx/OpenResty: The Upstream Keepalive Mistake

Most API Gateways in 2019 (Kong, 3scale, or raw Nginx) use Nginx under the hood. The most common configuration error I see is failing to enable keepalives to the upstream services.

By default, Nginx acts as a polite HTTP/1.0 client to your backend services. It opens a connection, sends the request, gets the response, and closes the connection. This means for every single API call, you are paying the price of a full TCP handshake (SYN, SYN-ACK, ACK) between the gateway and your microservice.

Pro Tip: TLS Handshakes are CPU expensive. Even on internal networks, if you are using mTLS between microservices, failing to reuse connections will skyrocket your gateway's CPU usage.

Here is the correct way to configure an upstream block in nginx.conf to reuse connections:

upstream backend_service {
    server 10.0.0.5:8080;
    server 10.0.0.6:8080;

    # The magic number. Keeps 32 idle connections open per worker.
    keepalive 32;
}

server {
    location /api/v1/ {
        proxy_pass http://backend_service;
        
        # Required for keepalive to work
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

Without the empty Connection header, Nginx will forward the "Close" header to the backend, defeating the purpose.

Data Privacy & Latency: The Norwegian Context

We are operating in a post-GDPR world. The Datatilsynet (Norwegian Data Protection Authority) is not lenient. When you tune your gateway, you often introduce caching to improve read performance. But where does that cache live?

If you are using a US-based cloud provider, even their "EU" zones can be legally ambiguous regarding data access (Schrems II is looming on the horizon, creating uncertainty). Storing cached API responses containing PII (Personal Identifiable Information) on a disk you don't fully control is a risk.

This is where local sovereignty meets performance. Hosting on CoolVDS in our Oslo facility ensures two things:

1. Legal Compliance: Data stays within Norwegian jurisdiction.
2. Latency: The round-trip time (RTT) from a user in Trondheim to Oslo is ~12ms. To Frankfurt, it's ~35ms. To US East, it's ~110ms. For an API gateway executing multiple serial requests, that physical distance destroys the user experience.

Storage I/O: Why NVMe Matters for Gateways

"But API Gateways are just CPU and RAM, right?"

Wrong. Logging. Access logs, error logs, audit trails. In a high-traffic environment, your gateway is writing gigabytes of text to disk every hour. If your VPS runs on standard SATA SSDs (or heaven forbid, spinning rust), your I/O wait times will spike during peak load. The kernel will block the Nginx worker process while it waits for the disk to confirm the write.

We benchmarked this. On a standard SSD, high-volume logging introduced a 4-8ms jitter on 99th percentile requests. On CoolVDS NVMe instances, that jitter vanished. NVMe allows for parallel command queues that standard AHCI controllers cannot handle. In 2019, if you aren't on NVMe, you are running legacy infrastructure.

The Final Configuration Check

Before you deploy, verify your SSL configuration. TLS 1.3 is now stable and supported in OpenSSL 1.1.1 (standard in Ubuntu 18.04). It reduces the handshake overhead significantly. Ensure your cipher suites prioritize performance without sacrificing security.

ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;
ssl_prefer_server_ciphers on;

Performance tuning is an iterative process. It requires measuring, tweaking, and measuring again. But you cannot tune your way out of bad hardware or a congested network. Start with a solid foundation. If you need a testbed that respects the physics of latency and the laws of data privacy, spin up a CoolVDS instance. It takes 55 seconds, and you'll feel the difference immediately.