You Are Losing 30% of Your Traffic to the Handshake

Let’s cut the marketing fluff. If your API Gateway takes more than 100ms to process a request before it even hits your backend microservices, you are bleeding users. In the Norwegian e-commerce sector, where mobile dominance is absolute, latency is the silent revenue killer. I’ve seen “optimized” architectures fail simply because the DevOps team treated the API Gateway as a black box rather than a high-performance proxy requiring surgical tuning.

In April 2021, the landscape is unforgiving. We aren't just battling for CPU cycles; we are battling the physical limits of TCP/IP and the legal complexities of GDPR following the Schrems II ruling. Whether you are running Kong, Tyk, or a raw NGINX reverse proxy, default settings are designed for compatibility, not speed. They are safe. They are slow.

Here is how we strip away the bloat and configure a gateway that screams, using the CoolVDS infrastructure as our reference baseline for hardware reality.

1. The Kernel is Your First Bottleneck

Most VPS providers hand you a standard Ubuntu 20.04 or CentOS 8 image and call it a day. The default Linux networking stack is tuned for a general-purpose file server, not an API Gateway handling 10,000 concurrent connections. Before you touch your application config, you must tune the TCP stack.

The net.core.somaxconn parameter defines the maximum number of connections queued for acceptance. The default is often 128. For an API Gateway, this is laughable. Under a connection storm (marketing push, Black Friday), your kernel will drop packets before NGINX even knows they exist.

Pro Tip: Never tune these values blindly. Monitor /proc/net/softnet_stat to see if your CPU is dropping packets at the network interface card (NIC) level. On CoolVDS KVM slices, we expose the necessary CPU flags to allow efficient packet steering.

The Essential sysctl.conf Tuning

Add these to your /etc/sysctl.conf to widen the highway:

# Maximize the backlog of pending connections
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535

# Allow reusing sockets in TIME_WAIT state for new connections
# Critical for high-throughput API gateways talking to local upstreams
net.ipv4.tcp_tw_reuse = 1

# Increase ephemeral port range (prevent port exhaustion)
net.ipv4.ip_local_port_range = 1024 65535

# Minimize TCP/IP overhead
net.ipv4.tcp_slow_start_after_idle = 0

Apply these with sysctl -p. Without this, your fancy API Gateway is trying to drink from a firehose through a straw.

2. NGINX & Kong: The Configuration Surgery

Most gateways today (Kong, APISIX) are built on top of OpenResty (NGINX + Lua). The bottleneck is rarely Lua; it's the NGINX worker configuration and upstream keepalives.

A common mistake I see in audits is the lack of keepalive connections to the upstream services. If your Gateway has to perform a full TCP handshake (SYN, SYN-ACK, ACK) with your backend microservice for every single API call, you are adding 3-5ms of latency per hop unnecessarily. Multiply that by 1,000 req/sec, and your CPU usage spikes just handling handshakes.

Optimizing the Upstream Block

Here is the correct way to configure an upstream block in NGINX to maintain persistent connections to your backend:

upstream backend_microservice {
    server 10.0.0.5:8080;

    # The secret sauce: Maintain 64 idle connections open
    keepalive 64;
}

server {
    location /api/v1/ {
        proxy_pass http://backend_microservice;
        
        # Required for keepalive to work
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        
        # Buffer tuning for JSON payloads
        proxy_buffers 16 16k;
        proxy_buffer_size 32k;
    }
}

If you are using Kong (v2.x), you need to adjust the nginx_http_keepalive_requests and upstream_keepalive_pool_size in your kong.conf. Default values are often too conservative for high-traffic environments.

3. The Hardware Reality: NVMe or Die

In 2021, if your hosting provider is still putting your API Gateway on SATA SSDs (or worse, spinning rust), move. Now. API Gateways logging access requests, rate-limiting counters, and caching responses generate massive random I/O.

In a shared hosting environment (