The "Default Config" is Your Enemy

It is November 2016. If you are still running your API Gateway on a default Nginx installation, you are practically asking for latency. With the explosion of microservices this year and the shift toward heavy mobile client usage, a 500ms overhead on the gateway is unacceptable. I recently audited a setup for a client here in Oslo who couldn't figure out why their shiny new REST API choked whenever concurrent connections hit 2,000. The code was efficient Go, but the gateway was stock Ubuntu 16.04 Nginx.

The bottleneck wasn't the CPU. It was file descriptors and a poorly configured TCP stack. Here is how we fixed it, and how you should configure your VPS Norway instances to handle high-throughput traffic without sweating.

1. The Foundation: File Descriptors

In Linux, everything is a file. A socket is a file. When Nginx acts as a reverse proxy, it opens two connections: one to the client, one to the upstream service. If your ulimit is the default 1024, you will hit a wall immediately.

First, check your limits:

If that says 1024, fix it. In /etc/security/limits.conf:

root soft nofile 65535
root hard nofile 65535
nginx soft nofile 65535
nginx hard nofile 65535

Then, tell Nginx to actually use them. This is the first thing I change in nginx.conf on any CoolVDS instance:

worker_rlimit_nofile 65535;

events {
    worker_connections 20480;
    use epoll;
    multi_accept on;
}

Pro Tip: Don't just set `worker_connections` to an arbitrary high number. It must be lower than `worker_rlimit_nofile` divided by `worker_processes`. If you overshoot, you'll see "Too many open files" in your error logs during peak loads.

2. Keepalive to Upstreams

This is the most common mistake I see in 2016. By default, Nginx speaks HTTP/1.0 to upstream servers and closes the connection after every request. This means for every single API call, you are doing a full TCP handshake (SYN, SYN-ACK, ACK) with your backend. That is wasted CPU and added latency.

You need to enable HTTP/1.1 and keepalive connections to your backends:

upstream backend_api {
    server 10.0.0.5:8080;
    # Keep 64 idle connections open to the backend
    keepalive 64;
}

server {
    location /api/ {
        proxy_pass http://backend_api;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

The empty Connection header is crucial. Without it, Nginx forwards the "Close" header from the client to the backend, defeating the purpose.

3. Kernel TCP Stack Tuning

Nginx can only be as fast as the OS kernel allows. The default Linux networking stack is tuned for generic desktop usage, not high-performance packet switching. On a low-latency network like we have here in Norway connecting to NIX, you want to fill the pipe.

Edit /etc/sysctl.conf. I've used these specific settings on CoolVDS NVMe instances to handle 20k req/sec spikes:

# Maximize the backlog of incoming connections
net.core.somaxconn = 65535

# Allow reusing sockets in TIME_WAIT state for new connections
net.ipv4.tcp_tw_reuse = 1

# Increase the range of ephemeral ports
net.ipv4.ip_local_port_range = 1024 65535

# Protect against SYN flood attacks
net.ipv4.tcp_syncookies = 1

# Fast Open (requires 3.7+ kernel, standard in Ubuntu 16.04)
net.ipv4.tcp_fastopen = 3

Apply these with sysctl -p. The tcp_tw_reuse flag is controversial to some, but in a controlled heavy-load environment, it prevents you from running out of sockets.

4. SSL/TLS Termination: Speed vs. Security

It's late 2016. If you aren't using HTTP/2, you are falling behind. HTTP/2 multiplexing solves the head-of-line blocking problem, but it requires encryption. SSL handshakes are expensive.

To optimize this, we prioritize ECDHE (Elliptic Curve Diffie-Hellman) which is faster than traditional RSA. We also must enable the SSL session cache to avoid re-negotiating the handshake for returning clients.

ssl_session_cache shared:SSL:50m;
ssl_session_timeout 1d;
ssl_session_tickets on;

# Modern cipher suite for late 2016 standards
ssl_protocols TLSv1.2;
ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256';
ssl_prefer_server_ciphers on;

Why Infrastructure Matters: The CoolVDS Reality

You can tune configurations all day, but software cannot fix slow I/O. When Nginx writes access logs or buffers large request bodies to disk, standard SATA SSDs become a bottleneck. This is effectively "iowait" stealing CPU cycles from your application.

At CoolVDS, we don't use legacy storage. Our instances run on local NVMe storage. Why does this matter for an API gateway? Because when you are logging thousands of requests per second for compliance (Datatilsynet requirements here in Norway are strict), that disk write needs to be instant. If the disk blocks, Nginx blocks. If Nginx blocks, your users see a spinner.

Local Latency and Legal Compliance

Hosting outside of Norway introduces two problems: latency and legal ambiguity. With the Privacy Shield agreement replacing Safe Harbor this year, data sovereignty is a hot topic. By hosting your API gateway in Oslo on CoolVDS, you slash round-trip times (RTT) to Norwegian ISPs to under 5ms. Furthermore, keeping logs within Norwegian borders simplifies GDPR compliance preparations significantly.

Final Thoughts

Performance isn't magic. It is the sum of a thousand small optimizations. By moving from default settings to a tuned stack on top of high-performance hardware, you can double your throughput without doubling your bill.

Don't let slow I/O kill your SEO. Deploy a test instance on CoolVDS in 55 seconds and see the difference NVMe makes.