Kill the VPN: Implementing Zero Trust Architecture on Linux in 2018

Date: July 26, 2018

Stop me if you've heard this one before: You have a strict firewall. You have a VPN. You think your internal network is a safe haven. Then one developer gets their laptop compromised at a coffee shop in Grünerløkka, the attacker rides the VPN tunnel into your "trusted" zone, and suddenly they have root access to your entire MySQL cluster.

The "Castle and Moat" strategy—where we harden the perimeter and trust everything inside—is dead. The Equifax breach last year taught us that. With the GDPR enforcement that kicked in this past May, relying on a perimeter firewall isn't just bad architecture; it's a liability that could cost you 4% of your global turnover.

In this guide, we are tearing down the perimeter. I'm going to show you how to implement Zero Trust principles on a Linux stack right now. No expensive proprietary appliances, just standard tools: Nginx, OpenSSL, and IPTables.

The Philosophy: Never Trust, Always Verify

Google has been publishing papers on BeyondCorp since 2014, but in 2018, most sysadmins are still manually managing IP whitelists. Zero Trust mandates that we shift access controls from the network perimeter to the individual device and user.

Does the web server really need to talk to the backup server? No. Does the developer need SSH access to production? Probably not.

War Story: I recently audited a client in Oslo who relied entirely on IP whitelisting for their Admin panel. They had a static IP at their office. Simple, right? Until their ISP rotated their IP block without warning during a maintenance window. The entire dev team was locked out for 4 hours. We switched them to client-side SSL certificates (mTLS) that afternoon. Authentication moved from the network layer to the application layer. Problem solved.

Step 1: Mutual TLS (mTLS) with Nginx

Forget passwords. Forget IP whitelists. The strongest way to secure an internal endpoint (like a Kibana dashboard or a staging API) in 2018 is Mutual TLS. The server authenticates the client via a certificate installed on their machine.

First, you need to act as your own Certificate Authority (CA).

1. Generate the CA and Keys

# Create the CA Key and Certificate
openssl genrsa -des3 -out ca.key 4096
openssl req -new -x509 -days 365 -key ca.key -out ca.crt

# Create the Server Key and CSR
openssl genrsa -out server.key 4096
openssl req -new -key server.key -out server.csr

# Sign the Server Certificate
openssl x509 -req -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt

# Create the Client Key and CSR (for your laptop)
openssl genrsa -out client.key 4096
openssl req -new -key client.key -out client.csr

# Sign the Client Certificate
openssl x509 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 02 -out client.crt

# Package Client Key for Browser Import (PKCS12)
openssl pkcs12 -export -out client.p12 -inkey client.key -in client.crt -certfile ca.crt

2. Configure Nginx to Demand the Certificate

Now, edit your nginx.conf. We aren't just enabling SSL; we are enabling ssl_verify_client.

server {
    listen 443 ssl;
    server_name internal-dashboard.coolvds.com;

    ssl_certificate /etc/nginx/certs/server.crt;
    ssl_certificate_key /etc/nginx/certs/server.key;
    
    # The Magic of Zero Trust
    ssl_client_certificate /etc/nginx/certs/ca.crt;
    ssl_verify_client on;

    location / {
        proxy_pass http://localhost:5601;
        # Pass details to backend if needed
        proxy_set_header X-Client-DN $ssl_client_s_dn;
    }
}

If a user tries to access this URL without the client.p12 installed in their browser or Keychain, Nginx drops the connection immediately. It doesn't matter if they have the password. It doesn't matter if they are on the "trusted" VPN. No cert, no entry.

Step 2: Micro-Segmentation with IPTables

If an attacker compromises your web server, can they pivot to your database? On a standard flat network, yes. In a Zero Trust model, we assume the web server is already compromised.

We need to lock down outbound traffic. The web server should only be allowed to talk to the specific private IP of the database on port 3306, and nothing else.

# Flush existing rules
iptables -F

# Default Policy: DROP EVERYTHING
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT # We will restrict this too, but be careful not to lock yourself out via SSH

# Allow established connections (crucial)
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Allow SSH only from specific admin VPN IP or Jump Host
iptables -A INPUT -p tcp --dport 22 -s 10.8.0.5 -j ACCEPT

# Allow Web Traffic
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Micro-segmentation: Allow loopback
iptables -A INPUT -i lo -j ACCEPT

# Log dropped packets (for debugging)
iptables -A INPUT -j LOG --log-prefix "IPTables-Dropped: "

Pro Tip: On CoolVDS instances, we provide a private network interface (usually eth1). Bind your database listeners ONLY to this private interface. Do not bind MySQL to 0.0.0.0.

Step 3: Identity-Aware SSH (The Bastion Host)

Managing SSH keys across a team of 10+ developers is a nightmare. Keys get lost, people leave the company, and keys remain authorized.

In 2018, Netflix is popularizing the concept of SSH Certificates (BLESS), but for smaller teams, a well-configured Bastion Host (Jump Box) is the pragmatic Zero Trust approach.

Configure your /etc/ssh/sshd_config on the target servers to ONLY accept connections from the Bastion's IP:

# /etc/ssh/sshd_config on Target Server
PasswordAuthentication no
PermitRootLogin no
AllowUsers deploy

# Only allow connection from the internal Bastion IP
Match Address 10.0.0.5
    PermitRootLogin no
Match All

This enforces a choke point. You can log, audit, and control access at the Bastion level. If a developer leaves, you revoke their access to the Bastion, and they are instantly locked out of the entire fleet.

The Infrastructure Reality: KVM vs. Containers

You cannot build a secure house on a sinking foundation. Zero Trust relies on isolation.

Many budget VPS providers in Europe are still using OpenVZ or LXC containers. While efficient, these share a kernel with the host. If there is a kernel panic or a container escape vulnerability (like the recent Dirty COW exploit variants), your neighbor's compromised instance becomes your problem.

This is why CoolVDS strictly uses KVM (Kernel-based Virtual Machine). With KVM, your OS has its own kernel, completely isolated from the host and other tenants. If you are serious about security—especially with Datatilsynet watching your GDPR compliance—you need hardware virtualization, not just containerization.

Why Location Matters

Latency is the enemy of security. If your security handshake takes 500ms, your developers will find a way to bypass it. Hosting your infrastructure in Norway (Oslo) ensures that local traffic stays local.

Furthermore, data sovereignty is critical. With the US CLOUD Act passed earlier this year, hosting data outside the EEA carries new risks. Keeping your encrypted data on NVMe storage within Norwegian borders is the safest bet for compliance.

Summary: The Checklist

Implementing Zero Trust isn't an overnight task. Start by isolating your most critical database. Set up mTLS for your internal admin dashboards. And ensure your underlying infrastructure isn't leaking data through a shared kernel.

Don't wait for a breach to rethink your architecture. Spin up a KVM instance on CoolVDS today and start building a network that assumes the worst—so you can deliver the best.