Stop Using FTP: Architecting a High-Performance Jenkins CI Pipeline in 2014

It is April 2014. If you are still deploying production code by dragging files from your local machine to a server via FileZilla, you are a liability to your company. I don't say this to be mean; I say it because I've seen too many config.php files overwritten at 5 PM on a Friday, taking down entire e-commerce sites.

The industry is shifting. We are moving toward Continuous Integration (CI) and Continuous Deployment (CD). Tools like Jenkins and Travis CI are no longer luxuries for Silicon Valley unicorns; they are requirements for any serious development shop in Oslo or Bergen. But here is the problem: Most of you are setting up your build servers on cheap, oversold OpenVZ containers, and then you wonder why your Java builds hang for 20 minutes.

I’m the guy they call when the server melts. Today, we are going to fix your pipeline, secure your data under Norwegian law, and talk about why hardware isolation (KVM) is the only virtualization that matters.

The Hidden Bottleneck: It’s Not CPU, It’s I/O Wait

When a developer pushes code to Git, your CI server wakes up. It checks out the code, downloads dependencies (Maven, Ruby Gems, npm modules), compiles binaries, runs unit tests, and packages the artifact.

Most sysadmins look at top and see high load averages, assuming they need more CPU cores. They are usually wrong.

In 90% of the cases I audit, the bottleneck is Disk I/O. Compiling code and installing thousands of small files (looking at you, node_modules) generates massive random Read/Write operations. On a shared hosting environment or a noisy VPS, your "guaranteed" CPU cycles are useless because the processor is waiting for the hard drive to catch up.

The "Noisy Neighbor" Effect

In 2014, many budget VPS providers use OpenVZ. This is container-based virtualization where the kernel is shared. If your neighbor on the physical node decides to run a heavy MySQL query or a Bitcoin miner, your disk performance tanks. You have no control over this.

This is why at CoolVDS, we strictly use KVM (Kernel-based Virtual Machine). KVM provides hardware virtualization. Your RAM is yours. Your kernel is yours. And most importantly, your I/O throughput is isolated. If you are building a CI/CD pipeline, do not settle for anything less than KVM.

Optimizing Jenkins for Speed and Stability

Let’s get technical. Assuming you are running Jenkins on an Ubuntu 12.04 LTS or the brand new 14.04 server, here is how we tune it for a high-performance environment.

1. Tuning the JVM

Jenkins runs on Java. Out of the box, the heap settings are conservative. If you have a CoolVDS instance with 4GB of RAM, don't let Jenkins guess how much to use. Force it.

Edit your configuration file:

sudo nano /etc/default/jenkins

Look for JAVA_ARGS. We want to increase the heap size and enable the concurrent garbage collector to prevent "stop-the-world" pauses during builds.

#/etc/default/jenkins

JAVA_ARGS="-Xmx2048m -XX:MaxPermSize=512m -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -Djava.awt.headless=true"

This configuration allocates 2GB specifically to Jenkins, ensuring it doesn't crash halfway through a Maven build.

2. Moving Build Workspaces to Tmpfs (Ramdisk)

If you have enough RAM, stop writing temporary build artifacts to disk. Write them to memory. This is the single biggest speed hack for CI pipelines in 2014. By mounting the Jenkins workspace in RAM, you eliminate disk I/O entirely for intermediate files.

Add this to your /etc/fstab:

tmpfs   /var/lib/jenkins/workspace   tmpfs   rw,size=1g,uid=jenkins,gid=jenkins,mode=0755   0   0

Mount it:

sudo mount -a

Pro Tip: Only do this if your build artifacts are archived elsewhere (like Artifactory) or deployed immediately. Data in tmpfs vanishes on reboot. This is perfect for the "Clean, Build, Test" lifecycle.

3. The Deployment Script

Forget FTP. We use rsync over SSH. It’s atomic, it’s secure, and it only transfers changed blocks. Here is a standard deployment script wrapper we use for staging environments:

#!/bin/bash
# deploy.sh

SRC="/var/lib/jenkins/workspace/MyProject/build/"
DEST="user@production-server:/var/www/html/"
EXCLUDE="--exclude '.git' --exclude 'config.php'"

# Ensure we are using strict checking to avoid MITM attacks
RSYNC_OPTS="-avz --delete -e 'ssh -o StrictHostKeyChecking=no'"

echo "Starting deployment to Norway Production..."

rsync $RSYNC_OPTS $EXCLUDE $SRC $DEST

if [ $? -eq 0 ]; then
    echo "Deployment Successful."
    # Optional: Reload Nginx/Apache
    ssh user@production-server "sudo service nginx reload"
else
    echo "Deployment Failed!"
    exit 1
fi

Data Sovereignty: Why Norway?

We need to talk about the elephant in the room: Edward Snowden. Since the leaks last year, every CTO in Europe is nervous about hosting data on servers owned by US companies, where the Patriot Act applies.

While the EU is currently debating the General Data Protection Regulation (a massive privacy reform that we expect to see finalized in the coming years), the current reality is that Datatilsynet (The Norwegian Data Protection Authority) is incredibly strict regarding personal data handling.

If you are developing software for Norwegian health services, finance, or even basic e-commerce, you cannot risk your data traversing US-controlled networks. Hosting your CI/CD pipeline and your staging servers within Norway—specifically connected to the NIX (Norwegian Internet Exchange)—ensures two things:

1. Legal Compliance: Your data stays within the jurisdiction of Norwegian courts.
2. Low Latency: If your dev team is in Oslo, ping times to a CoolVDS server in our local datacenter will be 2-5ms. Compare that to 40ms+ for hosting in Frankfurt or 100ms+ for US East. When you are typing commands in SSH all day, that lag adds up.

The Hardware Reality Check

In 2014, SSDs (Solid State Drives) are finally becoming affordable for enterprise use, but many providers still hoard them for "Premium" plans, leaving you with spinning SAS drives.

At CoolVDS, we made a strategic decision. We don't sell spinning rust for primary storage. All our VPS instances run on storage arrays backed by high-performance flash storage or high-speed RAID-10 architectures optimized for random I/O. When you combine this with the KVM virtualization I mentioned earlier, the difference is night and day.

Benchmark Comparison (Sysbench FileIO)

Conclusion: Build Fast, Fail Fast, Sleep Better

Automating your deployment pipeline is the single best investment you can make in your engineering culture. But automation requires reliable infrastructure. You cannot build a skyscraper on a swamp.

Stop fighting with noisy neighbors and sluggish hard drives. Treat your build server as a first-class citizen. By leveraging KVM isolation, proper JVM tuning, and local Norwegian hosting, you ensure that your team waits for code reviews, not progress bars.

Ready to fix your pipeline? Deploy a high-performance KVM instance on CoolVDS today. With our direct peering to NIX and pure hardware isolation, your builds will finish before you even have time to grab a coffee.