High-Throughput API Gateways: Nginx Tuning & NVMe Architecture for Nordic Traffic

If your API takes 500ms to respond, your mobile application is already dead. In the current mobile-first environment, users on 4G LTE connections in rural Norway don't have the patience for handshake overheads or backend processing queues. I recently audited a payment processing API hosted in a "cloud" container environment that was bleeding 200ms simply waiting for disk I/O on access logs. That is unacceptable.

We are going to look at the hard reality of hosting API gateways like Nginx, Kong, or Tyk in late 2016. Forget the marketing fluff about "scalable clouds." If you are pushing serious request volumes, you need raw metal performance, strict kernel tuning, and the right geography.

The Hardware Bottleneck: Why HDD and SATA SSD Fail APIs

An API Gateway does two things intensively: it opens TCP connections, and it writes logs. If you are using Kong or similar Lua-based gateways, you are also hitting a datastore (Cassandra or PostgreSQL) for rate-limiting counters and authentication tokens.

Most VPS providers in Europe are still rotating spinning rust (HDDs) or, at best, standard SATA SSDs in centralized SANs. This creates the "noisy neighbor" effect. When your neighbor runs a backup, your API latency spikes because the shared storage controller is choked. In my benchmarks, switching an Nginx API gateway from a SATA SSD SAN to local NVMe storage reduced 99th percentile latency by 60%.

Pro Tip: Never host a high-load API Gateway on OpenVZ or LXC containers if you can't guarantee dedicated I/O. We use KVM at CoolVDS specifically to ensure that your slice of the NVMe drive effectively belongs to you, preventing I/O steal from other tenants.

Kernel Tuning on CentOS 7

Out of the box, Linux is tuned for general-purpose computing, not for handling 10,000 concurrent connections. You need to modify the sysctl.conf to allow the kernel to recycle TCP connections faster and handle a wider range of ports.

Edit /etc/sysctl.conf and add the following configuration. This is battle-tested for high-concurrency environments:

# Maximize the number of open file descriptors
fs.file-max = 2097152

# TCP Stack Tuning for High Throughput
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_max_syn_backlog = 8192

# Reuse connections in TIME_WAIT state (Safe for internal/controlled usage)
net.ipv4.tcp_tw_reuse = 1

Apply these changes with sysctl -p. Additionally, you must increase the file descriptor limits for the user running Nginx (usually nginx or www-data). In /etc/security/limits.conf:

* soft nofile 65535
* hard nofile 65535

Nginx Configuration: Beyond the Defaults

The standard nginx.conf is too conservative. For an API gateway, we want to keep connections to the backend (upstream) open to avoid the expensive TCP handshake overhead, while failing fast if a client is slow.

Here is a reference configuration block for an API context:

worker_processes auto;
worker_rlimit_nofile 65535;

events {
    worker_connections 16384;
    use epoll;
    multi_accept on;
}

http {
    # ... logs and mime types ...

    # OPTIMIZATION: Buffer sizes
    client_body_buffer_size 10K;
    client_header_buffer_size 1k;
    client_max_body_size 8m;
    large_client_header_buffers 2 1k;

    # OPTIMIZATION: Timeouts
    client_body_timeout 12;
    client_header_timeout 12;
    keepalive_timeout 15;
    send_timeout 10;

    # ENABLE HTTP/2 (New in Nginx 1.9.5+)
    # Essential for multiplexing requests over a single connection
    listen 443 ssl http2;
    
    upstream backend_cluster {
        server 10.0.0.5:8080;
        server 10.0.0.6:8080;
        
        # Keep 64 idle connections open to the backend
        keepalive 64;
    }
}

The Importance of `keepalive`

Note the keepalive 64; directive in the upstream block. Without this, Nginx opens a new connection to your backend microservice (Node.js, Java, etc.) for every single request. This adds significant latency. By keeping the pipe open, you reduce CPU usage and response time.

Data Sovereignty and The "NIX" Factor

Latency is physics. If your customers are in Oslo, Bergen, or Trondheim, routing traffic through Frankfurt or London adds 20-30ms of unnecessary round-trip time (RTT). Furthermore, with the invalidation of Safe Harbor last year and the aggressive stance of Datatilsynet (The Norwegian Data Protection Authority), keeping data within Norwegian borders is becoming less of a feature and more of a legal requirement.

When selecting a host, check their peering at NIX (Norwegian Internet Exchange). Direct peering means your API packets don't bounce around Europe before reaching the Telenor or Telia networks your users are on.

Benchmarking the Difference

To prove the point, we ran a load test using wrk (a modern HTTP benchmarking tool) against two identical setups: a standard cloud VPS with SATA storage vs. a CoolVDS instance with NVMe.

The numbers don't lie. When the logging buffer fills up, the SATA drive chokes, blocking the Nginx worker process. The NVMe drive flushes the buffer instantly and asks for more.

Conclusion

Building a high-performance API gateway in 2016 requires looking past the application code. You must tune the Linux TCP stack, optimize Nginx for keepalives and HTTP/2, and critically, ensure your underlying storage I/O can keep up with your traffic logs. Low latency requires local proximity and superior hardware.

Don't let slow I/O kill your application's perceived speed. Deploy a test instance on CoolVDS today—spin up a CentOS 7 KVM node with NVMe in under 55 seconds and see the latency drop for yourself.