Console Login

GitOps in 2020: Stop Manually Touching Your Production Clusters

GitOps in 2020: Stop Manually Touching Your Production Clusters

If you are still SSH-ing into your servers to run docker-compose up or, god forbid, running kubectl edit deployment directly against your production cluster, you are part of the problem. I’ve spent the last decade cleaning up after "cowboy engineering" where a single typo brought down e-commerce sites during Black Friday. It ends today.

We are in 2020. The tools have matured. With Kubernetes 1.17 becoming the standard for stability and Helm 3 finally getting rid of Tiller, there is no excuse for drift. The concept is simple: Git is the single source of truth. If it isn't in the repo, it doesn't exist. This is the core of GitOps, and for those of us managing infrastructure in Norwayβ€”where Datatilsynet (The Norwegian Data Protection Authority) watches compliance like a hawkβ€”audit trails are not optional; they are survival.

The "Pull" Model: Why Your CI Should Not Touch K8s

In the traditional CI/CD pipeline (Jenkins, GitLab CI), the build server builds the artifact and then runs a script to push it to the cluster. This is dangerous. It requires giving your CI serverβ€”which often has a massive attack surfaceβ€”god-mode credentials to your production environment. If your Jenkins gets compromised, your entire infrastructure is gone.

GitOps flips this. We use a Pull Model. An operator living inside the cluster (like ArgoCD or Flux) watches the Git repository. When it sees a change in the manifest, it pulls it down and applies it. No external credentials needed. The cluster protects itself.

The Architecture

Here is the setup I deployed last week for a fintech client in Oslo, running on CoolVDS high-performance KVM instances:

  • Infrastructure: 3x CoolVDS NVMe Instances (Control Plane + Workers) running Ubuntu 18.04 LTS.
  • Orchestration: Kubernetes 1.17.
  • GitOps Controller: ArgoCD v1.4.
  • Secret Management: Bitnami Sealed Secrets.

Why CoolVDS? Because when you run a Kubernetes control plane, etcd is incredibly sensitive to disk latency. If your fsync latency spikes because your provider oversold their HDD storage, your cluster leader election fails, and your API goes down. CoolVDS guarantees NVMe I/O performance, which is critical for the stability of the GitOps operator constantly reconciling state.

Directory Structure Matters

Do not mix your application source code with your infrastructure manifests. You need a separation of concerns. I recommend the following repository structure:

β”œβ”€β”€ apps/
β”‚   β”œβ”€β”€ backend-api/
β”‚   β”‚   β”œβ”€β”€ base/
β”‚   β”‚   β”‚   β”œβ”€β”€ deployment.yaml
β”‚   β”‚   β”‚   β”œβ”€β”€ service.yaml
β”‚   β”‚   β”‚   └── kustomization.yaml
β”‚   β”‚   └── overlays/
β”‚   β”‚       β”œβ”€β”€ production/
β”‚   β”‚       β”‚   β”œβ”€β”€ kustomization.yaml
β”‚   β”‚       β”‚   └── patch-replicas.yaml
β”‚   β”‚       └── staging/
β”œβ”€β”€ infrastructure/
β”‚   β”œβ”€β”€ nginx-ingress/
β”‚   └── cert-manager/
└── cluster-config/

This structure leverages Kustomize (native in kubectl since 1.14), allowing you to keep a base configuration and patch it for environments. This keeps your YAML DRY (Don't Repeat Yourself).

Implementing ArgoCD (The 2020 Standard)

While Flux is great, ArgoCD offers a visual UI that helps non-ops developers understand what is happening. Here is how you define an Application in ArgoCD to sync your repo to your cluster.

apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: payment-gateway namespace: argocd spec: project: default source: repoURL: https://github.com/my-org/infra-manifests.git targetRevision: HEAD path: apps/backend-api/overlays/production destination: server: https://kubernetes.default.svc namespace: production syncPolicy: automated: prune: true selfHeal: true

Note the selfHeal: true flag. This is the magic. If a junior dev manually deletes a Service or changes a resource limit via the CLI, ArgoCD detects the drift immediately and reverts the change to match what is in Git. This enforces immutability.

Pro Tip: Never commit raw Secrets to Git. In 2020, the standard pattern is Sealed Secrets. You encrypt the secret on your laptop using a public key, commit the encrypted soup to Git, and the controller inside the cluster decrypts it using the private key. It’s GDPR compliant because the data at rest in the repo is unreadable.

Handling Database Migrations

The hardest part of GitOps is stateful data. You cannot just "revert" a database schema change. For our MariaDB clusters hosted on CoolVDS, we separate schema migrations from the deployment manifest.

We use a Kubernetes Job that runs helm upgrade hooks. Here is a snippet of how to ensure your migration runs before the new app pods start:

apiVersion: batch/v1
kind: Job
metadata:
  name: "{{ .Release.Name }}-db-migrate"
  annotations:
    "helm.sh/hook": pre-install,pre-upgrade
    "helm.sh/hook-weight": "-5"
    "helm.sh/hook-delete-policy": hook-succeeded
spec:
  template:
    spec:
      containers:
      - name: db-migrate
        image: "my-registry/db-migrate:v2.1"
        command: ["./migrate", "-database", "mysql://user:pass@db-host:3306/dbname"]
      restartPolicy: Never

This ensures that if the migration fails, the deployment aborts, and ArgoCD will mark the sync as failed, alerting your team via Slack.

Latency and Data Sovereignty

Hosting location matters. If your user base is in Scandinavia, hosting your Kubernetes nodes in Frankfurt or London adds unnecessary latency (20-30ms). By using CoolVDS, you are placing your workloads directly in Oslo. We see ping times to NIX (Norwegian Internet Exchange) as low as 1-2ms.

Metric CoolVDS (Oslo) US Cloud Provider (Frankfurt)
Latency to Oslo User ~2ms ~28ms
Disk I/O (Random 4k) High (Dedicated NVMe) Variable (Noisy Neighbors)
GDPR Compliance Native (Norway) Complex (Privacy Shield concerns)

Furthermore, keeping data within Norwegian borders simplifies compliance with local laws. You don't want to explain to your legal team why your customer data is bouncing through a router in Virginia.

Final Thoughts: The "Sleep Well at Night" Factor

GitOps isn't just a buzzword; it's an operational insurance policy. It documents your infrastructure history, enables instant rollback, and removes the human error element from deployments. But software is only as good as the hardware it runs on.

Running a GitOps workflow requires a control plane that doesn't choke under load. I've seen cheap VPS providers kill connections during a heavy Helm apply, leaving the cluster in a zombie state. Don't risk it.

Ready to build a resilient infrastructure? Spin up a KVM instance on CoolVDS today. With our pure NVMe storage and Oslo-based data centers, your Kubernetes cluster will be as fast as it is compliant.