Kill the Wait: Optimizing CI/CD Pipelines for High-Performance DevOps in 2016

The 20-Minute Coffee Break is Killing Your Company

I walked into a development shop in Oslo last week. Good team, smart guys. But every time a developer pushed to Git, they physically stood up and walked away. Why? Because their Continuous Integration (CI) pipeline took 24 minutes to build, test, and package a simple Magento application. That is unacceptable.

We are in 2016. The concept of "commit and wait" should be dead. If you are serious about DevOps, feedback loops need to be measured in seconds, not fractions of an hour. The problem usually isn't your code; it's your infrastructure gasping for air.

Let's look at why your builds are slow and how to fix it using tools available right now on CentOS 7 and Jenkins.

The Invisible Enemy: Disk I/O

Most people blame CPU when Jenkins lags. They throw more cores at the VM and wonder why the build time only drops by 5%. Here is the reality: CI/CD is a disk-heavy operation. npm install, composer update, checking out thousands of small files from Git, creating Docker images—these are IOPS (Input/Output Operations Per Second) killers.

On standard spinning HDD VPS hosting, your random write speeds are likely capping out around 100-150 IOPS. When five developers push code simultaneously, that queue explodes. You can see this clearly if you install iotop.

yum install iotop -y
sudo iotop -oPa

If you see your kworker or jbd2 (journaling block device) hitting 99% I/O wait, your storage is the bottleneck. This is where CoolVDS makes the difference. We use NVMe storage which offers upwards of 20,000 IOPS. In my testing, switching a Jenkins master from a standard SATA SSD to a CoolVDS NVMe instance cut a PHP/Composer build time from 12 minutes to 3 minutes flat.

Architecture: Stop Building on Master

A classic rookie mistake I see in setups across Europe is running builds directly on the Jenkins master node. The master should only orchestrate. The heavy lifting must happen on slave nodes (agents).

If the master goes down because a build exhausted the memory, your whole pipeline stops. Decouple them. Here is how we set up a robust slave node on a secondary CoolVDS instance using SSH.

1. Prepare the Slave Node (CentOS 7):

# Create a workspace user
useradd -m -d /var/lib/jenkins jenkins

# Install Java (Required for the slave agent)
yum install java-1.8.0-openjdk -y

# Ensure git is installed
yum install git -y

# Generate keys on Master and copy to Slave
ssh-copy-id jenkins@192.168.1.50

2. Configure Jenkins Master:

Go to Manage Jenkins > Manage Nodes > New Node. Set the launch method to "Launch slave agents via SSH". This keeps your master clean and responsive.

Dockerizing the Build Process

Docker is maturing fast (version 1.10 is solid). Instead of managing dependency hell on your slave nodes (conflicting Ruby versions, different Node.js versions), run your tests inside containers. This ensures environment parity between Dev, Stage, and Prod.

However, running Docker inside a virtual machine requires the kernel to support cgroups properly. Many budget VPS providers use OpenVZ, which often restricts Docker usage due to shared kernels. CoolVDS runs on KVM (Kernel-based Virtual Machine), giving you a dedicated kernel. This is non-negotiable for Docker stability.

Here is a snippet to run a clean build environment, mounting the workspace:

docker run --rm \
  -v "$WORKSPACE":/usr/src/app \
  -w /usr/src/app \
  node:5.9.0 \
  npm install && npm test

This spins up a Node 5.9 container, mounts your current Jenkins workspace, runs the install/test, and then destroys the container. No leftover garbage files.

Memory Management and Swap

Java is hungry. If your Jenkins instance crashes with OutOfMemoryError, you need to tune the heap. On a 4GB CoolVDS instance, don't let Java take it all; leave room for the OS.

Edit /etc/sysconfig/jenkins:

JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -Xmx2048m -XX:MaxPermSize=512m"

Pro Tip: Even with plenty of RAM, create a swap file. The Linux kernel moves stagnant pages to swap to free up RAM for disk caching (which speeds up Git and file operations).

dd if=/dev/zero of=/swapfile bs=1M count=2048
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile
echo '/swapfile none swap sw 0 0' >> /etc/fstab

Data Sovereignty: The Norwegian Advantage

With the Safe Harbor agreement invalidated last year, where you host your code and data matters more than ever. The Datatilsynet (Norwegian Data Protection Authority) is stringent. By hosting your CI/CD pipeline and artifacts in Norway, you are adding a layer of compliance safety that hosting in the US lacks right now.

Latency matters too. If your team is in Oslo or Bergen, pushing gigabytes of Docker images to a server in Frankfurt or Virginia is a waste of bandwidth. A local CoolVDS instance peering at NIX (Norwegian Internet Exchange) ensures your docker push happens at line speed.

The Bottom Line

Stop paying developers to stare at progress bars. High-performance CI/CD requires high-performance I/O and dedicated resources. You cannot achieve continuous deployment if your infrastructure is intermittent.

If you are ready to cut your build times in half, spin up a KVM instance on CoolVDS today. We have the NVMe storage your Jenkins server is begging for.