Stop Using kubectl apply: A Guide to GitOps Workflows on Kubernetes in 2019

It’s 3 AM. Your pager goes off. The Oslo cluster is down. Why? Because Steve from the frontend team manually edited a ConfigMap three days ago to "fix a quick bug," and today's automated deployment just overwrote it, crashing the pod loop. If this sounds familiar, you aren't managing infrastructure; you're managing chaos.

I’ve spent the last decade cleaning up after "cowboy engineering." In 2019, with tools like Kubernetes becoming the standard for container orchestration, there is absolutely no excuse for manual intervention in production environments. We call this new paradigm GitOps.

The concept is simple: Git is the single source of truth. If it's not in the repo, it doesn't exist. If you change it on the server, the automated agent reverts it. Ruthless consistency.

The Architecture of Truth

In a traditional CI/CD push model, your Jenkins or GitLab runner has cluster-admin credentials to push changes to your cluster. This is a security nightmare, especially if you are dealing with strict Datatilsynet requirements or handling sensitive Norwegian citizen data. One compromised CI server gives an attacker the keys to the kingdom.

GitOps flips this. We use a Pull Model. An agent inside the cluster pulls changes from Git. The cluster keys never leave the cluster.

The Stack

• Version Control: GitLab (Self-hosted or Cloud)
• Orchestrator: Kubernetes 1.15
• Sync Agent: Flux (by Weaveworks)
• Infrastructure: CoolVDS NVMe KVM Instances

Setting Up Flux on Kubernetes

Let's get technical. We aren't just talking theory. We are deploying Flux to watch a repository and sync manifests.

First, ensure your CoolVDS instance is ready. We prefer KVM over OpenVZ here because we need kernel-level control for Docker and Kubernetes networking (CNI). Once you have your cluster running (via kubeadm or similar), install Helm (v2.14 is the current stable choice, though we all hate Tiller).

Add the Flux chart repo:

helm repo add weaveworks https://weaveworks.github.io/flux
helm install --name flux \
--set git.url=git@gitlab.com:your-org/k8s-config.git \
--set git.branch=master \
--namespace flux \
weaveworks/flux

Once Flux is running, it generates an SSH key. You need to add this to your GitLab repository as a Deploy Key with write access (if you want Flux to update image tags).

kubectl -n flux logs deployment/flux | grep identity.pub

Pro Tip: If you are running on servers in Norway to minimize latency to the NIX (Norwegian Internet Exchange), ensure your git.pollInterval is set appropriately. You don't need to poll every 10 seconds if your commit velocity is low. Set it to 1m to save CPU cycles.

Handling Secrets without Leaking Them

The biggest pain point in GitOps is: "I can't commit secrets to Git." Correct. You cannot.

In 2019, the robust solution is Sealed Secrets by Bitnami. It allows you to encrypt a secret into a `SealedSecret` CRD that is safe to commit. Only the controller running inside your cluster can decrypt it.

Here is how you seal a database password for your MariaDB instance:

# Create a raw secret locally (do not commit this!)
echo -n "super-secure-password" > db-pass.txt
kubectl create secret generic my-db-pass --from-file=password=db-pass.txt --dry-run -o json > secret.json

# Seal it
kubeseal < secret.json > sealed-secret.json

# Now you can safely commit sealed-secret.json

When Flux pulls sealed-secret.json, the Sealed Secrets controller decrypts it and creates a native Kubernetes Secret object. Magic.

The Importance of Underlying Hardware

GitOps relies heavily on the Kubernetes control plane. The etcd database is constantly being hit with read/write operations as the state is reconciled. If your VPS has slow I/O, your API server starts timing out, and Flux fails to sync.

I've seen clusters fall apart because they were hosted on cheap, oversold HDD storage. etcd is extremely sensitive to disk latency.

This is why we build our infrastructure on CoolVDS. The NVMe storage guarantees that etcd commits state changes instantly. When you are automating infrastructure, latency isn't just an annoyance; it's a failure mode.

Dealing with "The Drift"

What happens when someone manually edits the cluster? Flux will fight them. This is good.

If you edit a deployment scale manually:

kubectl scale deployment/nginx --replicas=5

On the next sync loop, Flux looks at Git, sees replicas: 2, and scales it back down. This enforcement ensures that your documentation (Git) always matches reality. It forces your team to update the repo, not the server.

Structure Your Repository

Do not dump everything into root. For a clean Norwegian enterprise setup, separate your environments.

/releases
  /prod
    /oslo-dc
      deployment.yaml
      service.yaml
  /staging
    /frankfurt-dc
      deployment.yaml

Point your Flux instance to the specific path. This allows you to use a single repo for multiple clusters, promoting code reuse while maintaining isolation.

Conclusion

Manual deployments are a relic of the past. By adopting GitOps, you gain an audit trail (GDPR compliance becomes trivial when every change is a git commit), better security, and higher stability.

However, automation amplifies the need for reliable underlying infrastructure. A GitOps loop that hangs because the hypervisor is stealing CPU cycles is useless. Don't let slow hardware be the bottleneck in your automation pipeline.

Ready to build a cluster that actually stays up? Deploy a high-performance NVMe VPS on CoolVDS today and stop fearing Friday deployments.