Slashing Build Times: A Pragmatic Guide to CI/CD Optimization

There is nothing quite as soul-crushing as watching a progress bar crawl across a screen while a client waits for a hotfix. In the world of DevOps, latency isn't just a network statistic; it is lost revenue. If your Continuous Integration (CI) pipeline takes twenty minutes to run a suite of unit tests that should take two, you are bleeding money.

It is December 2015. We have tools like Jenkins, Travis CI, and the rapidly maturing Docker ecosystem. Yet, I still see teams in Oslo and Bergen running builds on oversold, sluggish hardware that chokes on simple I/O operations. With the recent invalidation of the Safe Harbor agreement by the ECJ just two months ago, where you host your build server is now as important as how you configure it. If your code touches Norwegian user data, you likely need that CI server on European soil.

Let's cut through the marketing noise. Here is how we optimize CI/CD pipelines for raw speed and reliability using technologies available today.

1. The I/O Bottleneck: Why SSD is Non-Negotiable

Most CI tasks are I/O bound, not CPU bound. npm install, composer update, and compiling Java artifacts involve writing thousands of tiny files. On a traditional spinning HDD, the seek times will destroy your performance.

I recently audited a Magento deployment pipeline for a retailer in Trondheim. Their build took 18 minutes. By moving the Jenkins workspace to an SSD-backed volume, we cut that to 6 minutes. No code changes. Just physics.

Pro Tip: Check your Disk Wait time. If you see high %wa in top, your storage is the problem.

The Tmpfs Hack

If you have RAM to spare, stop writing temporary build artifacts to disk entirely. Mount your build directories in RAM using tmpfs. This is volatile storage—if the server reboots, the data is gone—but for a CI build, that doesn't matter.

Add this to your /etc/fstab to turn a folder into a RAM disk:

# /etc/fstab
tmpfs   /var/lib/jenkins/workspace   tmpfs   rw,size=2G   0   0

Remount, and watch your I/O wait drop to zero.

2. Docker: The End of "It Works on My Machine"

Docker 1.9 was released last month (November 2015), and it is stabilizing fast. If you aren't wrapping your build environments in containers, you are wasting time debugging environment drift. Instead of maintaining a complex Jenkins server with fifty different versions of PHP and Python installed, use Docker to spin up ephemeral build agents.

However, Docker introduces overhead. The default device mapper storage driver on CentOS 7 can be slow. We recommend using overlayfs (available in newer kernels) or ensuring your backing volume is high-performance.

Here is a standard pattern for running a test inside a clean container, mapping the workspace volume:

docker run --rm \
  -v "$WORKSPACE":/usr/src/myapp \
  -w /usr/src/myapp \
  php:5.6-cli \
  php vendor/bin/phpunit

This ensures your tests run in a pristine environment every single time.

3. Caching is King: The Local Proxy

Every time you run a build, are you downloading the same libraries from the US West Coast? That is latency you cannot afford. The round-trip time (RTT) from Norway to US servers is often 100ms+. Multiply that by 500 dependencies.

Set up a local caching proxy. For Java, use Sonatype Nexus. For PHP/Node, a simple Nginx reverse proxy with caching enabled works wonders.

Nginx Caching Config Snippet

Here is how to cache external resources aggressively:

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:10m max_size=10g inactive=60m use_temp_path=off;

server {
    location / {
        proxy_cache my_cache;
        proxy_pass http://registry.npmjs.org;
        proxy_set_header Host registry.npmjs.org;
        proxy_cache_valid 200 302 60m;
        proxy_cache_valid 404 1m;
    }
}

Hosting this cache on a local CoolVDS instance within the same datacenter (or at least within Europe) drastically reduces network time.

4. The "Steal Time" Trap and Virtualization

If you are on a cheap VPS, run top and look at the %st (Steal Time) column. If this is above 0.0%, your hosting provider is overselling their CPU cores. Your compiler is waiting for other tenants to finish their work.

CI/CD processes are bursty. When they run, they need 100% of the CPU immediately. This is why we at CoolVDS utilize KVM (Kernel-based Virtual Machine) with strict resource guarantees. Unlike older OpenVZ containers where neighbors can cannibalize your RAM, KVM provides better isolation. For a build server, consistency is more important than raw burst speed.

5. Legal Reality: The Safe Harbor Fallout

Since the Schrems I ruling in October, relying on US-based hosting for data that might contain Norwegian PII (Personally Identifiable Information) is risky. Datatilsynet (The Norwegian Data Protection Authority) has been clear: the legal ground has shifted.

Migrating your CI/CD infrastructure to a Norwegian or European provider isn't just about latency anymore; it's about compliance. If your dump files or test databases contain real customer data (which they shouldn't, but let's be honest, it happens), storing them on a server under US jurisdiction is a liability.

Summary: The Optimization Checklist

• Storage: Move from HDD to SSD immediately. Consider CoolVDS NVMe-ready tiers if your I/O load is extreme.
• Memory: Use tmpfs for build artifacts that don't need to be saved.
• Virtualization: Avoid oversold OpenVZ providers. Stick to KVM.
• Network: Cache dependencies locally to avoid trans-Atlantic latency.
• Compliance: Host within the EEA to satisfy the post-Safe Harbor legal landscape.

A slow pipeline breaks developer flow. It encourages bad habits like skipping tests to "save time." Don't let your infrastructure dictate your code quality.

Ready to stop waiting? Deploy a high-performance Jenkins instance on CoolVDS today. Our KVM slices are provisioned in under 60 seconds.