You Are Losing Users in the Handshake

Let’s cut the marketing fluff. If your API Gateway adds more than 15ms of overhead to a request, your architecture is bleeding money. In the age of distributed microservices, the gateway is the gatekeeper. It handles authentication, rate limiting, and routing. But frequently, it becomes the choke point.

I recently audited a setup for a Fintech client based in Oslo. They were routing traffic through a standard hyperscaler region in Frankfurt, terminating SSL, and then forwarding requests to their backend. The latency was averaging 120ms. For a payment processor, that is an eternity. By optimizing their gateway configuration and moving the workload to local NVMe storage in Norway, we cut that down to 35ms.

Here is how we did it, using technology that actually works in production today, not theoretical beta-ware.

1. The OS Layer: Stop Ignoring sysctl.conf

Most DevOps engineers deploy a Docker container and assume the host OS will handle the networking magic. This is a fatal error. The default Linux kernel settings are tuned for general-purpose computing, not high-throughput packet switching. If you are running high-load API gateways (like Kong, NGINX, or HAProxy), you need to talk to the kernel.

When you handle thousands of concurrent connections, you will hit the ephemeral port limit or suffer from slow TCP connection recycling. Here is the baseline /etc/sysctl.conf configuration we apply on CoolVDS instances intended for high-traffic gateways:

# Increase the maximum number of open file descriptors
fs.file-max = 2097152

# Increase system-wide port range
net.ipv4.ip_local_port_range = 1024 65535

# Enable fast recycling of TIME_WAIT sockets (Critical for high request rates)
net.ipv4.tcp_tw_reuse = 1

# Max number of packets in the receive queue
net.core.netdev_max_backlog = 16384

# Increase the maximum number of connections allowed in the backlog
net.core.somaxconn = 65535

# Disable slow start after idle to maintain throughput
net.ipv4.tcp_slow_start_after_idle = 0

Pro Tip: Never just copy-paste sysctl settings without understanding them. For example, tcp_tw_recycle was removed in Linux 4.12, yet I still see people trying to enable it in 2022. Stick to tcp_tw_reuse.

2. NGINX: Upstream Keepalives Are Not Optional

Whether you are using raw NGINX or an abstraction like Ingress-NGINX on Kubernetes 1.24, the mistake is usually the same: creating a new TCP connection for every single request to the upstream service.

SSL handshakes are expensive. TCP handshakes are expensive. If your gateway terminates the client connection but opens a new connection to your backend microservice for every request, you are burning CPU cycles on your VDS unnecessarily. You need to enable keepalives to the upstream.

upstream backend_api {
    server 10.0.0.5:8080;
    
    # Keep at least 64 idle connections open to the backend
    keepalive 64;
}

server {
    location /api/ {
        proxy_pass http://backend_api;
        
        # Standard HTTP/1.1 is required for keepalive
        proxy_http_version 1.1;
        
        # Clear the Connection header to persist the link
        proxy_set_header Connection "";
    }
}

Without this configuration block, NGINX defaults to HTTP/1.0 and closes the connection after the proxy request. In our benchmarks on CoolVDS Standard instances, enabling this reduced internal latency by approximately 40%.

3. TLS: The Elephant in the Room

In 2022, there is no excuse for using RSA certificates with 4096-bit keys for standard web traffic. They are heavy on the CPU during the handshake. Switch to Elliptic Curve Cryptography (ECC). An ECDSA P-256 key provides security equivalent to RSA-3072 but is significantly faster to compute.

Furthermore, ensure you are prioritizing TLS 1.3. It reduces the handshake from two round trips to one. If your users are in Oslo and your server is in Oslo (as it should be), the RTT is low, but TLS 1.3 still makes the connection feel "snappier."

ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256;
ssl_prefer_server_ciphers on;

4. The Hardware Reality: Why Virtualization Type Matters

You can optimize your config files until your fingers bleed, but if your underlying host is oversold, it won't matter. This is the "noisy neighbor" effect.

Many budget providers use container-based virtualization (like OpenVZ) or heavily oversell CPU credits on Xen/KVM setups. When a neighbor on the same physical node decides to mine crypto or compile a massive kernel, your API Gateway suffers from "CPU steal." This manifests as unpredictable latency spikes—the worst nightmare for an SRE.

At CoolVDS, we strictly use KVM (Kernel-based Virtual Machine) with rigid resource guarantees. We combine this with local NVMe storage. Why NVMe for an API Gateway? Logs. High-traffic gateways write massive access logs. If your disk I/O blocks, your NGINX worker processes block. Slow I/O kills SEO and user experience.

Comparison: Request Processing Time

5. Data Sovereignty and Schrems II

We cannot talk about hosting in Europe in 2022 without addressing the legal landscape. The Schrems II ruling has made relying on US-owned cloud providers legally complex for handling EU citizens' data. By hosting your API Gateway on CoolVDS in Norway, you are adding a layer of compliance safety. Data stays here. It doesn't accidentally route through a switch in Virginia.

Final Thoughts

Performance is an aggregate of marginal gains. 5ms from the kernel, 10ms from the database connection pool, 20ms from the TLS handshake. It adds up. But the foundation is the infrastructure.

If you are tired of debugging latency spikes caused by your provider's noisy neighbors, it is time to move.

Stop guessing. Deploy a KVM-based, NVMe-powered instance on CoolVDS today and see what 0% CPU steal looks like on your monitoring dashboard.