Microservices in Production: Architecture Patterns That Won't Kill Your Latency

Everyone is breaking their monoliths. It is the trend of the decade. But few realize that by splitting a perfectly functional application into twenty fragmented services, you are trading code complexity for network complexity.

I watched a development team in Oslo recently try to migrate a solid Magento platform to a microservices architecture using a budget European cloud provider. They expected scalability. What they got was a latency jump of 400ms per request. Why? because when Service A calls Service B, and they both sit on overloaded hypervisors with "noisy neighbors," physics wins.

If you are deploying microservices in 2019 without a strategy for service discovery, circuit breaking, and underlying hardware isolation, you are building a distributed house of cards. Let's look at the patterns that actually work in production, and the infrastructure required to support them.

1. The API Gateway Pattern (The Bouncer)

Do not let your clients (mobile apps, single-page applications) talk directly to your backend microservices. It is a security nightmare and creates tight coupling. You need a gatekeeper.

In 2019, you might be tempted to use heavy, enterprise Java-based gateways. Don't. Unless you have a specific need for complex transformation logic, a properly tuned NGINX instance is faster, lighter, and easier to manage. It handles SSL termination, rate limiting, and routing so your microservices don't have to.

Here is a battle-tested NGINX configuration snippet for routing traffic to different upstream services based on the URI. This assumes you are running NGINX 1.15+.

http {
    upstream user_service {
        server 10.0.0.5:8080;
        server 10.0.0.6:8080;
        keepalive 64;
    }

    upstream order_service {
        server 10.0.0.10:4000;
        keepalive 64;
    }

    server {
        listen 80;
        server_name api.coolvds-client.no;

        # Security: Hide version info
        server_tokens off;

        location /users/ {
            proxy_pass http://user_service;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            proxy_set_header X-Real-IP $remote_addr;
        }

        location /orders/ {
            proxy_pass http://order_service;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
        }
    }
}

Pro Tip: Notice the keepalive 64; directive in the upstream block? Without this, NGINX opens and closes a new TCP connection for every single request to your microservice. That TCP handshake overhead destroys performance. Keep those connections open.

2. Client-Side Service Discovery (The Phonebook)

Hardcoding IP addresses in /etc/hosts or configuration files is suicide in a microservices environment. Services die. Services scale up. IPs change. You need a dynamic registry.

While Kubernetes (now at version 1.13) handles this internally with CoreDNS, many of us are still running hybrid environments or pure Docker Swarm. For these setups, HashiCorp Consul is the standard.

Instead of your app guessing where the database is, it asks Consul. Here is how you register a service using the raw HTTP API (useful for health check scripts):

curl --request PUT \
  --data '{
    "ID": "redis-primary",
    "Name": "redis",
    "Tags": ["primary", "v1"],
    "Address": "10.0.0.20",
    "Port": 6379,
    "Check": {
      "DeregisterCriticalServiceAfter": "90m",
      "TCP": "10.0.0.20:6379",
      "Interval": "10s"
    }
  }' http://localhost:8500/v1/agent/service/register

When you use CoolVDS, we recommend setting up a private network interface for this traffic. You don't want your service discovery gossip protocols traversing the public internet. Our KVM instances support private VLANs, which keeps this chatter secure and ultra-fast (sub-millisecond).

3. The Database-per-Service Dilemma

The golden rule of microservices is decoupling. If Service A and Service B share the same MySQL database tables, you haven't built microservices; you've built a distributed monolith. Each service needs its own datastore.

However, this multiplies your I/O requirements. Instead of one big database server, you now have ten smaller ones running MongoDB, PostgreSQL 11, and Redis. This is where storage performance becomes the bottleneck.

Most VPS providers sell you "SSD" storage, but they throttle the IOPS (Input/Output Operations Per Second). When ten microservices try to write logs and update records simultaneously, the disk queue spikes, and your application hangs.

This is a hardware problem, not a software problem.

The Infrastructure Reality Check

You cannot solve physical latency with code. If your target audience is in Norway, hosting your cluster in a massive data center in Virginia or even Frankfurt adds unavoidable latency. Light speed is finite.

• Latency to Oslo (from Frankfurt): ~25-30ms
• Latency to Oslo (from Oslo/CoolVDS): ~2-5ms

In a microservice call chain where A calls B calls C calls D, that latency compounds. A 30ms delay becomes 120ms just on network round-trips.

Furthermore, we are nearly a year into GDPR enforcement. The Norwegian Data Protection Authority (Datatilsynet) is not lenient. Keeping your data within Norwegian borders on Norwegian infrastructure is the safest bet for compliance, ensuring data sovereignty.

Deploying the Foundation

When we built the CoolVDS platform, we rejected OpenVZ and LXC containerization for our core offering. Why? Because in a container-based VPS, you share the kernel with every other customer on the node. If they crash the kernel, you go down. If they get DDOSed, your network suffers.

We use KVM (Kernel-based Virtual Machine). Each CoolVDS instance runs its own kernel. It is true hardware virtualization. Combined with local NVMe storage, it creates the stable, low-jitter environment microservices require.

Here is a quick Docker Compose v3 example to get a monitoring stack running on your node, because you can't improve what you don't measure:

version: '3'
services:
  prometheus:
    image: prom/prometheus:v2.7.1
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"
    network_mode: host

  node-exporter:
    image: prom/node-exporter:v0.17.0
    ports:
      - "9100:9100"
    network_mode: host
    pid: host

Use network_mode: host here to get accurate metrics from the host network stack. If you see high iowait in your graphs, your storage is too slow. If you see high steal time, your provider is overselling CPU.

Conclusion

Microservices offer agility, but they demand respect for the underlying infrastructure. You need robust patterns like Gateways and Discovery, but you also need raw power. Don't let a budget host's spinning disks or oversaturated uplinks be the reason your architecture fails.

If you are building for the Norwegian market, you need low latency and high IOPS. Deploy a test instance on CoolVDS today. Spin up a KVM node with NVMe in 55 seconds and ping it from Oslo. The numbers won't lie.