The I/O Bottleneck: Why Your Builds Are Rotting in the Queue

It is 2020, yet I still see senior engineers staring at a Jenkins console output, waiting 15 minutes for a simple npm install to complete. We treat compute resources as infinite, but we ignore the physics of storage. If you are relying on shared runners from the big public cloud providers, you are fighting a losing battle against "noisy neighbors" and capped IOPS.

In the Norwegian hosting market, where latency to Oslo is often excellent, the bottleneck has shifted. It is no longer network speed; it is disk I/O. When you run five concurrent Docker builds, the random read/write operations on a standard SATA SSD (or heaven forbid, a spinning HDD) will bring your pipeline to a halt.

I have spent the last month debugging a deployment pipeline for a FinTech client in Oslo. We reduced their build time from 24 minutes to 6 minutes. We didn't rewrite the app. We fixed the infrastructure. Here is exactly how we did it.

1. The Hidden Cost of the Docker Context

Every time you trigger a build, the Docker daemon needs the build context. If you are blindly copying your entire root directory, you are forcing the I/O subsystem to read thousands of irrelevant files.

The Mistake:

COPY . .

The Fix: Use a rigorous .dockerignore file. This is not optional. In 2020, ignoring .git, node_modules (local), and log files is baseline competence.

# .dockerignore
.git
node_modules
dist
coverage
*.log
.DS_Store

By preventing these files from being sent to the Docker daemon, we reduced the context size from 450MB to 2MB. On a remote VDS, that is a significant bandwidth and disk save.

2. Leveraging RAM Disks for Temporary Artifacts

Intermediate build artifacts often exist for seconds. Writing them to NVMe is fast, but writing them to RAM is instant. For heavy compile jobs (Java, C++, or even heavy Webpack builds), mounting a tmpfs volume can drastically reduce I/O wait.

If you control your runner infrastructure (which you should), you can mount the workspace in RAM. Here is how we configure it on a CoolVDS Linux instance:

# Create a 4GB RAM disk for build artifacts
sudo mount -t tmpfs -o size=4g tmpfs /var/lib/gitlab-runner/builds

Pro Tip: Be careful with persistent data. Only use tmpfs for scratch space. If your runner crashes, that data is gone. For a CI pipeline, this is usually a feature, not a bug, as it enforces clean builds.

3. Docker Layer Caching Strategy

You cannot rely on the registry to cache layers effectively if your Dockerfile is ordered poorly. The rule is simple: least frequently changed lines go first.

Here is the optimized structure we use for Node.js applications. Note the separation of dependency installation from code copying.

# 2020 Standard: Multi-stage build
FROM node:12-alpine AS builder

WORKDIR /app

# Copy only package files first to leverage cache
COPY package.json package-lock.json ./

# Install dependencies. 
# This layer is cached unless package-lock.json changes.
RUN npm ci --quiet

# Now copy the source code
COPY . .

# Build
RUN npm run build

# Production Stage
FROM nginx:1.17-alpine
COPY --from=builder /app/dist /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

By using npm ci instead of npm install, we get a deterministic install which is faster and safer for CI environments.

4. The "Self-Hosted" Advantage: Why Hardware Matters

This is where most teams fail. They use the default shared runners provided by SaaS CI platforms. These runners are usually throttled heavily.

The solution is spinning up your own GitLab Runner on a dedicated VDS. In Norway, strict data residency requirements (Datatilsynet) often mandate that code and PII do not leave the country. Hosting your runner on a CoolVDS instance in Oslo solves both the compliance issue and the performance issue.

Configuration: GitLab Runner with Docker Executor

Below is the config.toml optimization for a high-performance runner. We increase concurrency because our VDS instances run on KVM with guaranteed resources, not burstable credits.

concurrent = 4
check_interval = 0

[[runners]]
  name = "CoolVDS-Oslo-NVMe-Runner"
  url = "https://gitlab.com/"
  token = "PROJECT_TOKEN"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.docker]
    tls_verify = false
    image = "docker:19.03.11"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

Key configuration note: We map /var/run/docker.sock. This allows the container to spawn sibling containers rather than using the slower "Docker-in-Docker" (dind) service approach, though security implications must be managed within your team.

5. Monitoring I/O Wait (iowait)

How do you know if your current VPS is failing you? Check the wa (wait) metric in top or install iotop.

sudo apt-get install iotop -y
sudo iotop -oPa

If you see your disk utilization hitting 99% while the CPU is idling at 10%, your hosting provider is throttling your IOPS. This is common with budget providers who overload their host nodes. We strictly limit tenant density on CoolVDS hardware to ensure that when you pay for NVMe speed, you actually get it.

Conclusion: Take Control of Your Pipeline

Optimization is not just about writing better code; it is about understanding the underlying infrastructure. In 2020, there is no excuse for a 20-minute build time on a standard web application.

By implementing proper Docker caching, utilizing RAM disks, and moving away from oversubscribed shared runners to dedicated NVMe-backed instances, you save hours of developer time every week. Plus, keeping your data within Norwegian borders satisfies the legal department.

Ready to fix your build times? Spin up a CoolVDS instance with pure NVMe storage today. You can install a GitLab Runner and register it to your project in under 5 minutes.