Microservices in Production: Survival Patterns for 2019

Let’s get one thing clear immediately: splitting your monolithic application into twenty different Docker containers does not make you Netflix. In fact, if you do this without a rigorous understanding of network latency and infrastructure isolation, you are just building a distributed monolith that fails faster and harder than the legacy code you are trying to replace.

I have spent the last six months cleaning up a migration for a Nordic e-commerce client. They moved from a stable (albeit slow) Magento install to a microservices architecture on a budget public cloud provider. The result? Latency spikes of 400ms between services, timeouts during the Black Friday rush, and a database layer that choked on I/O. The code was fine. The architecture patterns and the underlying hardware were not.

In 2019, building microservices in Norway requires a balance of software patterns and hardware reality. We need to talk about the API Gateway, Service Discovery, and why standard SSDs aren't fast enough for distributed persistence.

1. The API Gateway Pattern: Your First Line of Defense

When you expose microservices directly to the client, you create a security nightmare and a chatty network. The standard approach now is the API Gateway. It handles SSL termination, rate limiting, and routing. While tools like Kong are gaining traction, good old Nginx remains the most battle-hardened tool for this job.

In a recent deployment, we used Nginx to handle traffic regulation before it ever hit our Kubernetes cluster. Here is the configuration we used to prevent the "thundering herd" problem when a service comes back online after a crash:

http {
    limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;

    upstream backend_services {
        # Using the least_conn algorithm is often better for microservices
        # than round-robin, as request times vary wildly.
        least_conn;
        server 10.0.0.5:8080 max_fails=3 fail_timeout=30s;
        server 10.0.0.6:8080 max_fails=3 fail_timeout=30s;
        keepalive 32;
    }

    server {
        listen 443 ssl http2;
        server_name api.coolvds-client.no;

        # SSL optimizations for lower latency handshake
        ssl_session_cache shared:SSL:10m;
        ssl_session_timeout 10m;

        location /orders/ {
            limit_req zone=api_limit burst=20 nodelay;
            proxy_pass http://backend_services;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            
            # Critical for microservices: Fail fast.
            # Don't let a hanging service hold up the Nginx worker.
            proxy_connect_timeout 5s;
            proxy_read_timeout 10s;
        }
    }
}

Notice the keepalive directive in the upstream block. Without this, Nginx opens a new connection to your microservice for every single request. In a high-traffic environment, you will exhaust the ephemeral port range on your server OS, leading to EADDRNOTAVAIL errors. This is a classic "works in dev, breaks in prod" scenario.

2. Service Discovery and The Kubernetes Reality

Hardcoding IP addresses is dead. In 2019, if you are not using dynamic service discovery, you are doing it wrong. While Consul is excellent, Kubernetes (k8s) has become the de-facto standard for container orchestration this year.

However, running k8s is resource-heavy. The control plane requires significant CPU, and `etcd` (the brain of the cluster) requires extremely low disk write latency. If `etcd` writes are slow, the entire cluster becomes unstable.

Here is a standard deployment snippet for a stateless microservice in Kubernetes 1.13, ensuring we have readiness probes configured so traffic isn't sent to a container that is still booting Spring Boot or Node.js:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: inventory-service
  labels:
    app: inventory
spec:
  replicas: 3
  selector:
    matchLabels:
      app: inventory
  template:
    metadata:
      labels:
        app: inventory
    spec:
      containers:
      - name: inventory-app
        image: registry.coolvds.com/inventory:v2.4.1
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "1Gi"
            cpu: "500m"
        readinessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 15
          periodSeconds: 5
        livenessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 45
          periodSeconds: 15

Pro Tip: Never deploy without resource limits. One memory leak in your image processing service can cause the Linux OOM (Out of Memory) Killer to terminate your database container if they share the same node. On CoolVDS instances, we recommend strict kernel-level isolation, but you must configure your manifests correctly.

3. The Database per Service Pattern & The I/O Bottleneck

This is where most architectures fail. The pattern dictates that each microservice owns its own data. This means instead of one giant database server, you might have five Postgres instances, three Redis caches, and a MongoDB cluster running simultaneously.

The I/O generated by this is massive. On standard cloud VPS hosting with "shared storage" (often Ceph or GlusterFS over a network), the latency kills you. Every transaction log write waits for the network. This is why we insist on local NVMe storage for database workloads.

Configuration for Performance

If you are running MySQL 5.7 or 8.0 for a microservice, you must tune InnoDB to respect the fact that it is not the only process on the server. However, you also need to ensure it uses the disk efficiently.

[mysqld]
# Ensure data integrity but try to group commits to reduce fsyncs
innodb_flush_log_at_trx_commit = 1

# Utilize NVMe IOPS capabilities
innodb_io_capacity = 2000
innodb_io_capacity_max = 4000
innodb_flush_method = O_DIRECT

# Buffer pool should be 70% of allocated RAM for that specific container/VM
innodb_buffer_pool_size = 2G

# Connection handling for microservices usually involves many small connections
max_connections = 500

4. The Norwegian Context: Latency and Compliance

Latency is physics. If your users are in Oslo, Bergen, or Trondheim, and your servers are in Frankfurt or Amsterdam, you are adding 20-40ms of round-trip time (RTT) to every request. In a microservices chain where Service A calls Service B which calls Service C, that latency compounds.

• Data Residency: With the implementation of GDPR last year and the strict stance of Datatilsynet, keeping customer data within Norwegian borders is not just a technical preference; for many sectors (finance, health), it is a legal safeguard.
• NIX (Norwegian Internet Exchange): Hosting locally means your traffic often routes through NIX, ensuring that data between your users (on Telenor or Telia) and your server stays within the country's backbone.

CoolVDS infrastructure is physically located in Norway. When you ping 127.0.0.1 from your app, it's fast. When you ping your user in Oslo from our datacenter, it's milliseconds, not tens of milliseconds. For a microservices architecture that requires constant chatter between nodes, this low latency is the difference between a snappy app and a sluggish one.

5. Infrastructure Isolation: KVM vs Containers

Finally, a word on the underlying metal. Many developers deploy Docker on top of cheap VPS providers that use OpenVZ or LXC. This is "containers on containers." It introduces noisy neighbor issues where another customer's high load steals CPU cycles from your Kubernetes scheduler.

The robust approach—and the one we enforce at CoolVDS—is using KVM (Kernel-based Virtual Machine). This provides hardware-level virtualization. Your Docker containers run inside a Linux kernel that you control, which sits on top of a hypervisor that guarantees your RAM and CPU are actually yours.

Summary Comparison

Microservices are complex enough without fighting your infrastructure. Don't let IO wait times or network latency be the reason your refactor fails. Build on a foundation that respects the physics of networking.

Ready to stabilize your cluster? Deploy a high-performance KVM instance in Oslo today and see the difference NVMe makes for your microservices.