Scaling the Gate: Advanced API Gateway Tuning for Low-Latency Norwegian Infrastructure

Let’s be honest: your API Gateway is probably the slowest component in your stack right now. You’ve optimized your microservices, you’ve indexed your Postgres database, but your edge layer is choking on 5,000 concurrent connections because you stuck with the default Linux limits. I see this constantly in audits across Oslo and Bergen. Developers deploy a standard Kong or NGINX instance on a budget VPS, and then panic when latency spikes during traffic surges.

It’s not always the code. It’s the physics of the underlying OS and the virtualization layer. If you are serving traffic to the Nordic market, latency matters. A request traveling from Tromsø to a data center in Frankfurt adds unnecessary milliseconds. Keep it local, keep it sharp.

This guide isn't about "digital transformation." It is about `sysctl.conf`, file descriptors, and making your API gateway scream.

1. The OS is Your Enemy (Until You Tune It)

Most Linux distributions shipping in 2018 come configured for general-purpose computing, not high-throughput network I/O. When your API gateway acts as a reverse proxy, it burns through ephemeral ports and file descriptors fast. If you see Too many open files or connection timeouts in your logs, you are hitting a hard ceiling.

First, check your current limits:

ulimit -n

If it says 1024, you are in trouble. For a production gateway on CoolVDS, we recommend bumping this significantly. You need to edit /etc/security/limits.conf to make it permanent:

* soft nofile 65535
* hard nofile 65535
root soft nofile 65535
root hard nofile 65535

But that’s just user-space. You also need to tell the kernel to allow more open files system-wide. Open /etc/sysctl.conf and add:

fs.file-max = 2097152

The TCP Stack Bottleneck

When an API gateway proxies a request, it opens a connection to the client and another to the upstream service. High churn rates leave thousands of sockets in the TIME_WAIT state. You need to recycle them faster. Add these lines to your sysctl config:

# Allow reusing sockets in TIME_WAIT state for new connections
net.ipv4.tcp_tw_reuse = 1

# Increase the range of ephemeral ports
net.ipv4.ip_local_port_range = 1024 65535

# Max number of packets in the receive queue
net.core.netdev_max_backlog = 16384

# Max SYN backlog
net.ipv4.tcp_max_syn_backlog = 8192

# Increase TCP buffer sizes for high-speed NVMe storage IO
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

Apply these changes with sysctl -p. If you are running on a provider that restricts kernel tuning (common with older OpenVZ setups), you are stuck. This is why we use KVM at CoolVDS—you get a dedicated kernel to tune exactly how you need it.

2. NGINX: The Engine Room

Whether you use raw NGINX, OpenResty, or Kong (which is built on NGINX), the core configuration principles remain identical. The default nginx.conf is often too conservative.

Here is a battle-tested configuration snippet for a gateway handling high concurrency. Note the worker_rlimit_nofile directive; it must exceed your worker_connections.

worker_processes auto;
worker_rlimit_nofile 65535;

events {
    multi_accept on;
    worker_connections 16384;
    use epoll;
}

http {
    # Optimization for zero-copy file transfer
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    
    # Keepalive tuning to reduce SSL handshake overhead
    keepalive_timeout 65;
    keepalive_requests 100000;
    
    # Buffer sizes - crucial for handling large API payloads without disk I/O
    client_body_buffer_size 128k;
    client_max_body_size 10m;
    client_header_buffer_size 1k;
    large_client_header_buffers 4 4k;
    
    # ... logs and other settings
}

Pro Tip: If you are serving an API, disable access logs for static assets or successful 200 OK health checks. Writing to disk for every health check kills I/O throughput, even on SSDs. On CoolVDS NVMe instances, you have high IOPS headroom, but why waste it on noise?

3. Upstream Keepalives: The Silent Killer

A very common mistake I see in code reviews is failing to enable keepalives to the upstream application servers (Node.js, Go, etc.). Without this, NGINX opens a brand new TCP connection for every single API call it proxies. That is expensive.

Configure your upstream block properly:

upstream backend_api {
    server 10.0.0.5:8080;
    server 10.0.0.6:8080;
    
    # Maintain 64 idle connections to the backend
    keepalive 64;
}

server {
    location /api/ {
        proxy_pass http://backend_api;
        
        # Required for HTTP 1.1 keepalive
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

4. TLS 1.3: It is Time (Late 2018 Update)

With OpenSSL 1.1.1 released recently, TLS 1.3 is finally becoming viable for production. It reduces the handshake latency by one full round-trip. For mobile users on spotty 4G networks in rural Norway, this latency reduction is noticeable.

If your repo supports it, enable it. If not, ensure you are at least using optimized cipher suites for TLS 1.2 to prioritize speed without sacrificing security.

ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;

5. The Hardware Reality: Noisy Neighbors and IO Wait

You can have the most perfectly tuned NGINX config in the world, but if your virtualization platform has "noisy neighbors" stealing CPU cycles, your p99 latency will be garbage. This is the fundamental flaw of cheap shared hosting.

When an API Gateway performs SSL termination, it is CPU intensive. When it caches responses or writes logs, it demands low-latency I/O. If your provider overcommits resources, your `iowait` spikes while the physical disk services someone else's database backup.

Why architecture matters here:

• KVM Virtualization: Ensures strict isolation of RAM and CPU instructions. No one steals your cycles.
• NVMe Storage: Traditional SATA SSDs cap out around 500 MB/s. NVMe drives, which we treat as standard at CoolVDS, push gigabytes per second with drastically lower seek times. This is critical for high-volume logging and caching layers.
• Network Proximity: For Norwegian businesses, hosting in Oslo or nearby ensures your data stays within the jurisdiction (GDPR compliance helper) and ping times remain minimal.

Datatilsynet (The Norwegian Data Protection Authority) has been very clear about data sovereignty since GDPR kicked in this May. Hosting your API gateway on a server physically located in Norway simplifies your compliance map significantly compared to routing traffic through a US-owned cloud region.

Final Thoughts

Performance tuning is an iterative process. Start by benchmarking your current setup with a tool like wrk:

wrk -t12 -c400 -d30s https://your-api.com/endpoint

Apply the kernel tweaks, adjust your file descriptors, and run it again. The difference is usually stark. Don't let your infrastructure be the reason your application feels slow.

Ready to see what your code can actually do when the hardware gets out of the way? Spin up a CoolVDS NVMe instance in Oslo today and test your throughput against your current provider.