Console Login

GitOps Workflow Best Practices: Automating Integrity from Oslo to the Edge

GitOps Architecture: The End of "It Works on My Machine"

If you are still running kubectl apply -f from your laptop, you are a liability. I’ve seen production clusters in Oslo go dark for hours because a junior dev manually patched a ConfigMap three months ago, and today's automated deployment wiped that undocumented fix. Configuration drift is not just annoying; it is an outage waiting to happen.

By mid-2021, the debate is effectively over. GitOps is not just a buzzword; it is the standard for managing complex infrastructure. For us operating in the Nordic region, it solves two critical problems: consistency across environments and strict auditability required by Datatilsynet (The Norwegian Data Protection Authority). When the state of your infrastructure mirrors the state of your Git repository, you eliminate the "mystery" of server configuration.

The Core Principle: Git as the Single Source of Truth

The concept is simple but ruthless. The Git repository is the only place where your infrastructure definition lives. You do not touch the server via SSH. You do not edit settings in a web UI. You make a Pull Request. If it's not in Git, it doesn't exist.

In a typical Norwegian enterprise setup, where we adhere to strict GDPR and Schrems II compliance, this provides an immutable log of who changed what and when. No more guessing why the latency to the database spiked at 03:00 AM.

Push vs. Pull: Why We Choose Pull

Traditional CI/CD (Jenkins, GitLab CI) uses a "Push" model. The pipeline has credentials to your cluster and pushes changes. This is a security risk. If your CI server is compromised, your production environment is exposed.

We prefer the "Pull" model using an operator inside the cluster, like Argo CD (which recently hit v2.0, adding significant stability). The cluster pulls its configuration from Git. It only needs read access to the repo, and no external service needs admin rights to your Kubernetes API.

Implementation: The Stack

For a robust setup on CoolVDS KVM instances, we standardize on the following stack:

  • Infrastructure Provisioning: Terraform (v1.0 is finally GA as of June 2021).
  • Kubernetes Distribution: K3s or Vanilla K8s (v1.21).
  • GitOps Controller: Argo CD.
  • Secret Management: Bitnami Sealed Secrets.

1. The Application Manifest

Here is how we define an application in Argo CD. Note the selfHeal policy. This ensures that if someone manually changes a value on the cluster, Argo CD immediately reverts it back to the Git state.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: nordic-payment-gateway
  namespace: argocd
spec:
  project: default
  source:
    repoURL: 'git@github.com:coolvds-ops/payment-gateway.git'
    targetRevision: HEAD
    path: k8s/overlays/production-oslo
  destination:
    server: 'https://kubernetes.default.svc'
    namespace: payments
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

2. Managing Infrastructure with Terraform

Before you can apply K8s manifests, you need the servers. Terraform allows us to define our CoolVDS instances as code. We use the local provider or generic Libvirt connectors where specific APIs aren't exposed, ensuring we aren't locked into proprietary cloud CLIs.

A typical main.tf for a high-availability control plane in 2021 looks like this:

terraform {
  required_version = ">= 1.0.0"
  required_providers {
    libvirt = {
      source = "dmacvicar/libvirt"
    }
  }
}

resource "libvirt_domain" "k8s_master" {
  name   = "k8s-master-01"
  memory = "4096"
  vcpu   = 2

  network_interface {
    network_name = "default"
  }

  disk {
    volume_id = libvirt_volume.os_image.id
  }

  # Cloud-init to bootstrap the node
  cloudinit = libvirt_cloudinit_disk.commoninit.id

  console {
    type        = "pty"
    target_port = "0"
    target_type = "serial"
  }
}
Pro Tip: When using Terraform on CoolVDS, always separate your state file storage. Use a remote backend (like an S3-compatible bucket or Consul) with state locking enabled. If two engineers run terraform apply simultaneously on local state, you will corrupt your infrastructure.

The Compliance & Performance Edge

This is where hardware reality meets software theory. You can have the best GitOps pipeline in the world, but if your underlying VPS has noisy neighbors or high I/O wait, your application will crawl.

Schrems II and Data Residency

Since the Schrems II ruling last year (July 2020), transferring personal data to US-owned cloud providers is legally perilous for Norwegian companies. By hosting your Kubernetes nodes on CoolVDS, you ensure data sovereignty. Our datacenters are in Europe, governed by European law. There is no ambiguous "cloud act" access.

NVMe Latency and Etcd

Kubernetes relies heavily on etcd for state storage. Etcd is incredibly sensitive to disk write latency. If fsync takes too long, the cluster leader election fails, and your GitOps syncs will time out.

We see this constantly with budget VPS providers using shared spinning rust (HDDs) or network-throttled SSDs. For a stable GitOps workflow, we utilize CoolVDS instances with direct-attached NVMe storage. Benchmarks in 2021 show NVMe drives delivering 300,000+ IOPS compared to the 500 IOPS of a standard SATA SSD.

To verify your disk latency for etcd, run this fio command on your node:

fio --rw=write --ioengine=sync --fdatasync=1 --directory=test-data --size=22m --bs=2300 --name=mytest

If the 99th percentile latency is above 10ms, your cluster is unstable. On our platform, we typically see sub-1ms results.

Handling Secrets without Leaking Them

You cannot commit raw passwords to Git. For 2021, the battle-tested solution is Bitnami Sealed Secrets. It uses asymmetric cryptography. The controller running in your cluster has the private key. You have the public key.

You encrypt the secret locally:

kubeseal --format=yaml --cert=pub-cert.pem < secret.yaml > sealed-secret.yaml

This sealed-secret.yaml is safe to commit to GitHub. Even if your repo is public, no one can decrypt it except the controller running inside your secure CoolVDS environment.

Conclusion

GitOps transforms operations from a sequence of frantic manual commands into a calm, engineering discipline. It aligns perfectly with the need for stability and compliance in the Norwegian market.

However, automation amplifies the underlying platform's characteristics. Automated garbage on a slow server is still garbage. You need raw compute power and I/O speed to match the velocity of your pipeline.

Don't let slow I/O kill your reconciliation loops. Deploy your GitOps stack on CoolVDS NVMe instances today and experience the stability of true high-performance hosting.