Stop Bleeding Cash on Slow Builds: CI/CD Pipeline Optimization Strategy

There is nothing more demoralizing for a senior engineer than staring at a blinking cursor while npm install fetches the entire internet or a Docker build hangs on a layer extraction. In the Nordic dev scene, we often over-engineer our application architecture but leave our build infrastructure running on under-powered, choked hardware. It makes no sense.

If your pipeline takes longer than 5 minutes to deploy a hotfix, your infrastructure is actively fighting your business logic. I’ve seen teams in Oslo burn thousands of kroner in developer hours simply waiting for shared runners on major cloud providers to spin up. The bottleneck is rarely the code; it’s the underlying metal.

Let’s cut through the marketing noise. We are going to look at the raw mechanics of disk I/O, network latency relative to NIX (Norwegian Internet Exchange), and kernel tuning to fix this. We are building for speed.

The Hidden Killer: Disk I/O Wait (iowait)

CI/CD is fundamentally a disk-abuse operation. You clone git repositories, you extract archives, you compile binaries, and you layer Docker images. These are random read/write heavy operations. Most generic VPS providers oversell their storage, putting you on standard SSDs shared by hundreds of neighbors. When a neighbor runs a backup, your build fails or stalls.

In 2022, running a build server on anything less than NVMe is negligence. You need high IOPS (Input/Output Operations Per Second) to handle the concurrent read/writes of a modern microservices build.

Pro Tip: Don't trust the label "SSD." Verify the throughput yourself. A high-performance build server should sustain random write speeds above 200MB/s without sweating.

Here is how I benchmark a potential CI runner before I even install the Docker daemon. I use fio to simulate a build workload:

sudo apt-get install fio -y

# Simulate a heavy build process: Random Read/Write, 4GB file, Direct I/O
fio --name=build_sim \
    --ioengine=libaio \
    --rw=randrw \
    --bs=4k \
    --direct=1 \
    --size=4G \
    --numjobs=4 \
    --runtime=60 \
    --group_reporting

If your IOPS are below 10k or your latency climbs above 10ms during this test, terminate the instance. It will choke under a real GitLab Runner load. This is why we standardized on KVM-based NVMe architecture at CoolVDS; container-based virtualization (LXC/OpenVZ) often exposes you to the "noisy neighbor" effect where stole CPU cycles kill your compilation times.

Network Latency: The Oslo Advantage

For Norwegian teams, data sovereignty and latency go hand in hand. If your artifacts are stored in `us-east-1` but your deployment target is a VPS in Oslo, you are adding unnecessary seconds to every fetch. Furthermore, with the Schrems II ruling casting shadows over data transfer to US-owned clouds, keeping your build artifacts (which contain your intellectual property) on Norwegian soil satisfies both the Datatilsynet and your need for speed.

Hosting your CI runner locally means you are millisecond-close to your production environment. But you also need to tune the network stack for the high churn of TCP connections that CI tools generate.

Kernel Tuning for CI Workloads

Default Linux kernels are tuned for long-lived connections (web servers), not the rapid-fire connections of a build script. Update your /etc/sysctl.conf to handle the load:

# /etc/sysctl.conf

# Reuse connections in TIME_WAIT state for new connections
net.ipv4.tcp_tw_reuse = 1

# Increase the range of ephemeral ports
net.ipv4.ip_local_port_range = 1024 65000

# Maximize the backlog of pending connections (crucial for parallel tests)
net.core.somaxconn = 65535

# Increase TCP buffer sizes for high-throughput artifact uploads
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

Apply these with sysctl -p. I have seen this simple change reduce random connection timeouts in Jenkins pipelines by 40%.

Optimizing the Docker Daemon

Most pipelines today are Docker-in-Docker or socket-binding. The default Docker configuration is conservative. To really fly, you need to enable BuildKit (standard in newer versions but ensure it's active) and configure the storage driver properly. If you are on a CoolVDS KVM slice, you have full control over the kernel, so use the overlay2 driver.

Check your /etc/docker/daemon.json. We also want to ensure we aren't throttled on log generation during verbose builds.

{
  "storage-driver": "overlay2",
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  },
  "default-ulimits": {
    "nofile": {
      "Name": "nofile",
      "Hard": 64000,
      "Soft": 64000
    }
  },
  "features": {
    "buildkit": true
  }
}

Restart Docker: systemctl restart docker. The ulimits setting is critical; builds often crash mysteriously because they hit the default open file limit of 1024. Bumping this to 64k solves those "phantom failures."

The Architecture: Dedicated Runners vs. Shared

Shared runners are fine for hobbyists. For professional teams, they are a security risk and a performance bottleneck. By deploying a dedicated runner on a CoolVDS instance, you gain:

Configuration for GitLab Runner

If you are using GitLab, the concurrent setting in config.toml is your throttle. On a 4 vCPU CoolVDS instance, setting this too high causes context switching lag. Setting it too low wastes the CPU.

concurrent = 4
check_interval = 0

[[runners]]
  name = "CoolVDS-Oslo-NVMe-01"
  url = "https://gitlab.com/"
  token = "YOUR_TOKEN"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "docker:20.10.16"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
    shm_size = 0

Notice the mapping of docker.sock. This allows the container to spawn sibling containers rather than using the slower Docker-in-Docker (dind) approach, though it has security implications you must weigh. For internal trusted builds, the speed gain is massive.

Conclusion

Optimizing CI/CD is about removing friction. The friction of rotational media, the friction of network distance, and the friction of resource contention. You cannot optimize a pipeline running on bad infrastructure.

Stop accepting slow builds as a fact of life. Spin up a CoolVDS instance in our Oslo data center, apply the kernel tunings above, and watch your deployment times drop. Your developers will thank you, and your CFO will appreciate the fixed costs.

Ready to fix your pipeline? Deploy a high-performance NVMe instance on CoolVDS today and get back to shipping code.