GitOps Is More Than Just 'git push': Hard Lessons from the Norwegian Trenches

I have watched too many production clusters burn because someone thought kubectl apply -f . from their laptop was a valid deployment strategy. It works fine on localhost until it deletes the ingress controller on production at 3 AM. If you are managing infrastructure in 2024 without a Single Source of Truth (SSOT), you are essentially gambling with your uptime.

We are going to talk about GitOps. Real GitOps. Not just keeping your YAMLs in a repo, but the actual pull-based architecture where your cluster synchronizes itself. We will focus on the Norwegian context—because latency to NIX (Norwegian Internet Exchange) and compliance with Datatilsynet matters more than you think.

The Architecture: Pull vs. Push

In the traditional "Push" model (Jenkins, GitLab CI pushing directly), your CI pipeline needs cluster-admin credentials. This is a security nightmare. If your CI runner gets compromised, your entire infrastructure is gone.

In the "Pull" model (GitOps), the cluster has an agent (like ArgoCD or Flux) that watches the Git repository. The cluster pulls changes. The CI system never touches the Kubernetes API. This is the only way I deploy now.

The Stack for May 2024

• Orchestrator: Kubernetes 1.29 (Stable, reliable).
• GitOps Agent: ArgoCD v2.10.
• Secret Management: Bitnami Sealed Secrets.
• Infrastructure: CoolVDS NVMe Instances (Oslo).

Step 1: Establishing the Control Plane

You need a stable environment for your GitOps controller. If the node hosting ArgoCD suffers from CPU steal (common on oversold budget VPS providers), your synchronization loops lag. You want the reconciliation to be instant.

On a standard CoolVDS KVM instance, I start by bootstrapping the cluster. We aren't using managed K8s here; we want raw control. Here is the K3s initialization optimized for low-latency networking:

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server \
  --disable traefik \
  --flannel-backend=host-gw \
  --tls-san=gitops.norway.local" sh -

Pro Tip: Using --flannel-backend=host-gw is faster than VXLAN encapsulation, but it requires Layer 2 connectivity between nodes. This works beautifully on CoolVDS private networking but fails on public clouds that block L2. Know your underlying metal.

Step 2: Deploying ArgoCD

Once the API is up, we install ArgoCD. Do not use the raw manifest from the internet blindly. We need to tune the Redis cache to handle high-frequency git polling.

helm repo add argo https://argoproj.github.io/argo-helm
helm upgrade --install argocd argo/argo-cd \
  --namespace argocd \
  --create-namespace \
  --set server.extraArgs={--insecure} \
  --set controller.resources.limits.cpu="1000m" \
  --set controller.resources.limits.memory="2Gi" \
  --set redis.resources.limits.memory="512Mi"

Why the resource limits? Because when you have 500 microservices trying to sync simultaneously, an unconstrained controller will eat your node's RAM and trigger the OOMKiller. I have seen it happen during Black Friday traffic surges.

Step 3: The Application Manifest

This is where the magic happens. You define the state of your application in a YAML file. ArgoCD ensures the cluster matches this state. If a developer manually deletes a deployment, ArgoCD recreates it immediately. It is self-healing infrastructure.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: nordic-payment-gateway
  namespace: argocd
spec:
  project: default
  source:
    repoURL: 'git@gitlab.com:coolvds-ops/payment-gateway.git'
    targetRevision: HEAD
    path: k8s/overlays/oslo-prod
  destination:
    server: https://kubernetes.default.svc
    namespace: payments
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Setting selfHeal: true is the difference between sleeping through the night and getting paged. It forces the cluster to revert any manual changes that deviate from Git.

The Norwegian Compliance Problem (GDPR & Data Residency)

Here is where things get messy for us in Europe. If your Git repository contains configuration data (ConfigMaps), and that repo is hosted on GitHub (US servers), are you compliant with Schrems II? Maybe. Maybe not.

The safest bet—and the one I recommend to CTOs paranoid about the Datatilsynet—is self-hosting your Git instance. We run a GitLab CE instance on a separate CoolVDS Storage VPS within Norway.

Latency Matters: When your K8s cluster (in Oslo) pulls from GitLab (in Oslo), the sync latency is sub-millisecond. When it pulls from US-East, you add 90ms+ to every reconciliation loop. Over thousands of syncs, this adds up to slow deployment pipelines.

Handling Secrets without Leaking Them

You cannot commit secrets.yaml to Git. That is a firing offense. In 2024, the standard is Sealed Secrets. You encrypt the secret on your laptop using the cluster's public key. Only the controller inside the cluster can decrypt it.

1. Install the client:

brew install kubeseal # Or use wget for linux binary

2. Generate the sealed secret:

kubectl create secret generic db-creds \
  --from-literal=password='SuperSecurePassword123!' \
  --dry-run=client -o yaml > secret.yaml

kubeseal --format=yaml --cert=pub-cert.pem \
  < secret.yaml > sealed-secret.yaml

You can safely commit sealed-secret.yaml to a public repository. It is useless without the private key stored safely on your CoolVDS master node.

Monitoring the GitOps Workflow

Blindly trusting automation is foolish. You need observability. I configure Prometheus ServiceMonitors to scrape ArgoCD metrics. We look for one specific metric: argocd_app_sync_status.

Why Infrastructure Integrity is Critical

GitOps is heavy on I/O. The controller is constantly cloning repos, verifying diffs, and hitting the K8s API. I have tried running this stack on cheap shared hosting where the "vCPU" is a slice of a slice. The result? workqueue_depth spikes, deployments stall, and developers start yelling.

We use CoolVDS because the NVMe storage throughput ensures that etcd (the brain of Kubernetes) never bottlenecks. If etcd slows down, your entire GitOps workflow grinds to a halt. When you are deploying a hotfix for a payment gateway, you do not want to wait for a "noisy neighbor" to finish their video rendering task.

Final Thoughts

GitOps forces discipline. It removes the "cowboy engineering" of manual patches. It makes your infrastructure auditable, which keeps the legal team happy, and resilient, which keeps the ops team happy.

But software is only as good as the hardware it runs on. Do not let slow I/O kill your agile workflow. If you are ready to build a pipeline that actually works, get a serious environment.

Deploy your GitOps control plane on a CoolVDS NVMe instance today and stop fighting your infrastructure.