Slashing CI/CD Build Times: Why Your VPS Infrastructure is the Bottleneck

I recently watched a senior developer stare at a Jenkins progress bar for 45 minutes. He wasn't coding. He wasn't solving problems. He was waiting for npm install and a Docker build to finish on an overloaded virtual machine. In an era where we aim for continuous deployment, waiting nearly an hour for feedback isn't just inefficient; it's a morale killer.

Most teams try to solve this by refactoring their build scripts. They tweak their Gruntfile, they mess with parallel jobs, or they blame the network. But 90% of the time, the bottleneck isn't the script. It's the infrastructure underneath it. If you are running your pipelines on oversold, noisy-neighbor cloud instances, you are fighting a losing battle against I/O latency.

The Silent Killer: I/O Wait

In May 2017, the shift to containerization is undeniable. We are all moving to Docker. But layering file systems (AUFS, Overlay2) on top of virtualized storage creates a massive I/O tax. When twenty developers push code simultaneously, and your CI server tries to extract node modules or compile Java artifacts, the disk queue depth explodes.

I saw this firsthand last month helping a frantic e-commerce agency in Oslo. Their deployments were timing out. They thought it was a PHP 7.1 configuration issue. It wasn't.

We ran iotop and saw the truth immediately:

Total DISK READ: 0.00 B/s | Total DISK WRITE: 34.50 M/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
 1245 be/4 jenkins     0.00 B/s   12.40 M/s  0.00 %  99.99 % java -jar jenkins.war
 3391 be/4 root        0.00 B/s   22.10 M/s  0.00 %  85.60 % dockerd

99.99% I/O Wait. The CPU was idle, waiting for the disk to catch up. They were hosting this on a budget "cloud" provider using standard SATA SSDs over a network storage protocol (Ceph/GlusterFS) that was saturated.

Hardware Matters: The NVMe Difference

This is where the choice of hosting becomes architectural, not just financial. For high-churn workloads like CI/CD pipelines, IOPS (Input/Output Operations Per Second) are the only metric that matters. Standard SSDs cap out around 5,000-10,000 IOPS. A proper NVMe drive, which talks directly to the PCIe bus rather than through a SATA controller, can handle 300,000+ IOPS.

At CoolVDS, we enforce a strict policy: KVM virtualization only, backed by local NVMe storage. We don't use network-attached storage for root volumes. This reduces latency from milliseconds to microseconds. When your CI pipeline is unzipping thousands of small files (looking at you, node_modules), that difference cuts build times in half.

Benchmarking Your Current Pipeline Host

Don't take my word for it. SSH into your current CI runner and run a simple fio test to simulate random write performance, which mimics a build process:

# Install fio on Ubuntu 16.04
sudo apt-get update && sudo apt-get install -y fio

# Run a random write test
fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=4k --direct=0 --size=512M --numjobs=1 --runtime=240 --group_reporting

If your IOPS are under 15,000, your infrastructure is throttling your developers.

Optimizing the Pipeline: 2017 Best Practices

Once you have the hardware (like a CoolVDS NVMe instance), you need to tune the software. Here are the specific configurations we use to keep pipelines flowing efficiently.

1. Docker Layer Caching

Rebuilding the entire image on every commit is wasteful. Organize your Dockerfile so that the package installation happens before the code copy. This utilizes the Docker cache for the heavy lifting.

# BAD PRACTICE
FROM node:6.10
COPY . /app
WORKDIR /app
RUN npm install
CMD ["npm", "start"]

# GOOD PRACTICE (Leverages Cache)
FROM node:6.10
WORKDIR /app
# Copy package.json first
COPY package.json /app/
# Install dependencies (cached unless package.json changes)
RUN npm install
# Then copy source code
COPY . /app
CMD ["npm", "start"]

2. Kernel Tuning for Heavy Workloads

Default Linux kernel settings aren't designed for the rapid creation and destruction of TCP connections seen in CI/CD. We add the following to /etc/sysctl.conf on our build agents:

# Increase the range of ephemeral ports
net.ipv4.ip_local_port_range = 1024 65000

# Reuse connections in TIME_WAIT state
net.ipv4.tcp_tw_reuse = 1

# Increase max open files (essential for Jenkins + Docker)
fs.file-max = 2097152

Apply these with sysctl -p.

The Norwegian Context: Latency and Sovereignty

Speed isn't just disk I/O; it's network latency. If your dev team is in Oslo or Bergen, pushing huge Docker images to a server in Virginia or Frankfurt adds unnecessary delay. The round-trip time (RTT) matters.

Pro Tip: Hosting in Norway isn't just about speed. With the GDPR enforcement date set for May 2018, the Datatilsynet is becoming stricter about where data lives. Keeping your artifacts and test databases (which often contain scrubbed but sensitive data) on Norwegian soil simplifies your compliance roadmap significantly.

Jenkins Pipeline as Code

If you are still using the Jenkins UI to configure jobs, stop. Jenkins 2.0 introduced Pipelines, and defining your build in a Jenkinsfile is the only way to manage complexity. Here is a snippet of a declarative pipeline we use for PHP projects, leveraging Docker agents:

pipeline {
    agent {
        docker {
            image 'php:7.1-cli'
            args '-v /var/run/docker.sock:/var/run/docker.sock'
        }
    }
    stages {
        stage('Build') {
            steps {
                sh 'composer install --no-interaction --optimize-autoloader'
            }
        }
        stage('Test') {
            steps {
                sh './vendor/bin/phpunit'
            }
        }
    }
    post {
        always {
            archiveArtifacts artifacts: 'build/libs/**/*.jar', fingerprint: true
        }
    }
}

This approach isolates the build environment. However, spinning up these containers requires the KVM hardware isolation we discussed earlier. On OpenVZ or shared containers, nested Docker capabilities are often restricted or unstable.

Conclusion: Stop Waiting, Start Shipping

Your developers cost significantly more than your servers. Saving $20 a month on a cheap VPS that wastes 10 hours of developer time a month is bad math. The combination of high-frequency CPUs and local NVMe storage is the only logical path for a CI/CD infrastructure that stays green.

If you are ready to stop watching progress bars and start merging PRs, benchmark your current setup. Then, compare it to a KVM instance optimized for heavy I/O.

Need a build server that keeps up with your team? Deploy a high-performance CoolVDS instance in Oslo today and see the difference in your build times.