Console Login

Stop Waiting for Jenkins: Accelerating CI/CD Pipelines with NVMe & Private Runners in Norway

The "Coffee Break" Deployment Syndrome

It’s 2 PM on a Tuesday. You pushed a critical hotfix to `master`. Twenty minutes later, you are still staring at a spinning wheel in Jenkins or GitLab CI. The status? "Pending allocation" or worse, crawling through `npm install` at 2MB/s.

If this sounds familiar, your infrastructure is bottlenecking your engineering team. In the fast-paced development environments of 2019, waiting for builds is not just annoying; it is a direct hit to TCO (Total Cost of Ownership). When you calculate the hourly rate of a senior developer in Oslo, spending 4 hours a week waiting for pipelines equates to thousands of kroner burned monthly.

The culprit is rarely the code. It is the infrastructure underneath the runner. Most SaaS CI providers place your jobs on oversubscribed containers fighting for disk I/O. Here is how we fix it using dedicated KVM instances and proper kernel tuning.

Why Disk I/O is the CI/CD Bottleneck

Continuous Integration is essentially a disk-torture test. Consider a standard pipeline:

  1. git fetch (Disk Write)
  2. docker pull (Network + Disk Write)
  3. npm install / mvn install (Massive Small Random I/O)
  4. Artifact archiving (Disk Read/Write)

On a standard HDD-based VPS or a shared runner, IOPS (Input/Output Operations Per Second) hit a ceiling immediately. During a heavy `node_modules` extraction, disk latency spikes. If you are sharing that physical disk with fifty other noisy neighbors, your build halts.

Pro Tip: Always check your iowait. If you are seeing values above 5% during a build, your CPU is sitting idle waiting for the disk. This is money wasted on compute you cannot use.

Architecture: The Private Runner Pattern

For Norwegian teams adhering to strict data sovereignty (keeping code within borders due to GDPR and Datatilsynet regulations), the hybrid approach is superior. Use your SaaS git provider (GitHub/GitLab) for code hosting, but delegate the heavy lifting to a private runner hosted in Norway.

Step 1: The Infrastructure

You need a clean environment with high-speed local storage. This is where CoolVDS fits the architectural diagram. Unlike standard cloud instances that throttle IOPS based on volume size, our NVMe-backed KVM slices offer consistent high throughput. This reduces the `npm install` phase from minutes to seconds.

Step 2: OS Tuning for Docker Workloads

A fresh Ubuntu 18.04 install isn't optimized for the hundreds of containers a CI server spawns daily. You need to adjust the kernel parameters.

Open /etc/sysctl.conf and apply these settings to handle high network traffic and file handle exhaustion—common failures during parallel testing.

# /etc/sysctl.conf optimization for CI Runners

# Increase the number of file watches (critical for heavy JS/Webpack builds)
fs.inotify.max_user_watches = 524288

# Increase open file limits
fs.file-max = 2097152

# Optimization for frequent short-lived connections (Docker NAT)
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_tw_reuse = 1

# Reduce swap usage to prioritize RAM for caching
vm.swappiness = 10

Apply changes with:

sudo sysctl -p

Implementing a GitLab Runner with Docker Executor

Let's set up a robust runner on a CoolVDS instance. We assume you have Docker installed (version 18.09 is the current stable standard).

1. Install the Runner

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
sudo apt-get install gitlab-runner

2. Registration & Configuration

Register the runner against your repo. When prompted, choose docker as the executor. This ensures every build runs in a clean, isolated container.

sudo gitlab-runner register \
  --non-interactive \
  --url "https://gitlab.com/" \
  --registration-token "YOUR_TOKEN_HERE" \
  --executor "docker" \
  --docker-image "docker:stable" \
  --description "norway-nvme-runner-01" \
  --tag-list "nvme,docker,norway"

3. Optimizing config.toml

The default configuration is too conservative. We want to utilize the full power of the VPS. Edit /etc/gitlab-runner/config.toml:

concurrent = 4
check_interval = 0

[[runners]]
  name = "norway-nvme-runner-01"
  url = "https://gitlab.com/"
  token = "YOUR_TOKEN"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "docker:stable"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

Critical Detail: Mapping /var/run/docker.sock allows for "Docker-in-Docker" capabilities without the overhead of the `dind` service, though it has security implications. For a private, trusted runner, this socket binding often yields the best performance for building images.

Benchmark: Shared vs. CoolVDS Private Runner

We ran a standard `create-react-app` build and test cycle. The results highlight the impact of storage latency.

Metric Shared SaaS Runner CoolVDS (4 vCPU / 8GB RAM NVMe)
Environment Setup 45s 12s
Dependency Install 142s 38s
Build & Test 210s 115s
Total Time ~6.5 Minutes ~2.7 Minutes

The Sovereignty Advantage

Beyond raw speed, hosting your CI infrastructure in Norway addresses the growing concerns around data privacy. With the legal landscape shifting (post-GDPR), ensuring that your intellectual property and temporary build artifacts reside on servers physically located in Oslo or nearby regions reduces compliance friction.

CoolVDS infrastructure is built with this specific requirement in mind. Low latency to NIX (Norwegian Internet Exchange) means your deployment scripts push to production servers in milliseconds, not seconds.

Final Thoughts

You cannot optimize a pipeline if the foundation is crumbling. Scripts and caching strategies can only go so far. Eventually, you need raw iron that can handle the I/O throughput of modern DevOps.

Stop letting your developers wait. Spin up a dedicated CI/CD node on CoolVDS today and turn that 20-minute wait into a 3-minute victory lap.