Your Gateway is the Choke Point. Fix It.

Most DevOps engineers spend weeks optimizing database queries and refactoring microservices, only to throw their work behind a default NGINX or Kong configuration that throttles traffic like a bottleneck on the E18 during rush hour. I’ve seen it happen. In a recent audit for a fintech client based in Stavanger, we found their application logic was processing requests in 12ms, but the gateway was adding a staggering 45ms overhead per request. That is unacceptable.

In the Norwegian market, where connectivity via NIX (Norwegian Internet Exchange) offers some of the lowest latencies in Europe, introducing software drag is a sin. If you are routing traffic from Oslo to Frankfurt or Stockholm, every millisecond of processing time counts against your total round-trip time (RTT).

This isn't a beginner's guide. We are going to tweak the Linux kernel, strip down SSL handshakes, and discuss why the underlying hardware of your "Managed VPS" is usually the silent killer of performance.

1. The Kernel: Open the Floodgates

Stock Ubuntu 24.04 LTS is designed for general-purpose computing, not for handling 50,000 concurrent API connections. Before you even touch your gateway software, you must tune the TCP stack. The default backlog queue is too small for high-burst traffic, leading to dropped SYN packets before your application even knows a user is trying to connect.

You need to modify /etc/sysctl.conf. These settings are aggressive but necessary for high-throughput API gateways.

# /etc/sysctl.conf

# Increase the maximum number of open file descriptors
fs.file-max = 2097152

# Maximize the backlog queue
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535

# Expand the port range for outgoing connections (crucial for proxying)
net.ipv4.ip_local_port_range = 1024 65535

# Reuse Time-Wait sockets immediately
net.ipv4.tcp_tw_reuse = 1

# BBR Congestion Control (Standard in Kernel 6.8+)
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr

Apply these with sysctl -p. If you are running on a legacy hypervisor that doesn't allow kernel parameter modification, you need to migrate. This is why we enforce KVM virtualization at CoolVDS; you get a dedicated kernel, not a containerized slice where sysctl is read-only.

2. SSL/TLS: The CPU Vampire

In 2025, TLS 1.3 is mandatory. However, the handshake process is CPU intensive. If your VPS provider is oversubscribing CPUs (selling the same core to 10 customers), your SSL termination will lag during peak hours due to "CPU Steal." This is the percentage of time your virtual CPU waits for the physical CPU to serve it.

Check your steal time immediately:

top -b -n 1 | grep "Cpu(s)" | awk '{print $16, "Steal Time"}'

If that number is above 1.0%, move your workload. For configuration, ensure you are utilizing OpenSSL 3.x with assembly optimizations enabled. In NGINX, optimize the session cache to prevent repeated handshakes for returning clients:

http {
    # Cache SSL sessions for 10 minutes. 
    # 10m size can hold approx 40,000 sessions.
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;

    # Disable session tickets if you have multiple gateway nodes 
    # without a shared key rotation mechanism, otherwise keep on.
    ssl_session_tickets on;

    # Buffer size optimization
    ssl_buffer_size 4k;
}

Pro Tip: Setting `ssl_buffer_size` to 4k (default is often 16k) reduces the Time To First Byte (TTFB). Smaller buffers mean the first chunk of data is sent to the client faster, rather than waiting to fill a large buffer. This is critical for mobile networks in rural Norway where bandwidth fluctuates.

3. Upstream Keepalives: The Forgotten Flag

The most common configuration error I see is NGINX closing the connection to the backend service after every single request. This forces your gateway to open a new TCP connection to your microservice for every API call, doubling the overhead.

You must configure the `upstream` block to keep connections open. This is how you achieve true low latency:

upstream backend_api {
    server 10.0.0.5:8080;
    server 10.0.0.6:8080;

    # Keep 64 idle connections open to the backend
    keepalive 64;
}

server {
    location /api/ {
        proxy_pass http://backend_api;

        # Required for keepalive to work
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

4. The I/O Bottleneck: Logging

API Gateways generate massive amounts of logs. Every request writes an access log. On standard SATA SSDs, or worse, network-attached block storage with low IOPS limits, disk blocking can stall the entire NGINX worker process.

To mitigate this, use buffer logging:

access_log /var/log/nginx/access.log combined buffer=32k flush=5s;

This tells NGINX to hold logs in RAM and write them to disk only when the buffer is full or every 5 seconds. However, the real solution is hardware. You need NVMe storage. At CoolVDS, we use local NVMe arrays rather than remote network storage for our high-performance instances. The difference in I/O wait times is night and day.

Why Infrastructure Dictates Performance

You can tune configurations all day, but you cannot tune your way out of noisy neighbors or throttled hardware. When handling sensitive data subject to GDPR or waiting for Datatilsynet audits, you need predictability.

In a shared hosting environment, a neighbor running a crypto miner affects your API latency. In a proper KVM environment with dedicated resource allocation, your performance baseline is flat and stable. Stability is the only metric that matters for an API gateway.

Don't let slow I/O or CPU steal kill your SEO rankings or user experience. Deploy a test instance on CoolVDS in 55 seconds and run `ab` (Apache Benchmark) against your current provider. The numbers will speak for themselves.