Taming the Hydra: Managing Docker at Scale in 2014

So, Docker 1.0 was released two weeks ago. The hype train has officially left the station. If you attended DockerCon or have been following the mailing lists, you know the promise: "Build once, run anywhere." It is a beautiful lie.

Sure, it runs anywhere—provided "anywhere" is a Linux kernel version 3.8+ with AUFS support and you don't care about persistent storage yet. But for those of us actually waking up at 3 AM when the pager goes off, the question isn't how to run a container. It's how to manage fifty of them across three different web nodes without losing our minds.

We are seeing a fragmentation in the ecosystem. You have the heavyweights like Apache Mesos, the new minimalist contenders like CoreOS Fleet, and the Google engineers teasing this "Kubernetes" project (which looks promising but is strictly pre-alpha right now). For a production environment in Norway today, what do you actually use?

The Contender: Apache Mesos + Marathon

If you need to run web-scale infrastructure like Airbnb or Twitter, Mesos is currently the only adult in the room. It abstracts CPU, memory, and storage away from machines (slaves) and presents them as a single pool of resources.

We recently migrated a high-traffic Magento setup to a Mesos cluster. The resilience is incredible, but the learning curve is a vertical wall. You need Zookeeper for state, Mesos Masters, Mesos Slaves, and then a framework like Marathon to actually schedule the Docker containers.

Here is what a typical JSON submission to the Marathon API looks like to launch a simple Nginx container:

curl -X POST -H "Content-Type: application/json" http://10.0.0.1:8080/v2/apps -d '{
    "id": "frontend",
    "cmd": "nginx -g \"daemon off;\"",
    "cpus": 0.5,
    "mem": 64.0,
    "instances": 3,
    "container": {
        "type": "DOCKER",
        "docker": {
            "image": "dockerfile/nginx"
        }
    }
}'

The Verdict: Powerful, but heavy. Running a Zookeeper quorum just to manage five web servers is overkill. Plus, the JVM overhead on the masters is real.

The Rebel: CoreOS & Fleet

On the other end of the spectrum is CoreOS. They are taking a radical approach: a minimal OS that updates itself (A/B partitioning) and uses systemd for everything. Their orchestration tool, Fleet, is essentially "distributed systemd."

If you know how to write a systemd unit file, you know Fleet. It relies on etcd (a distributed key-value store) to coordinate. It is lightweight and feels much more "Linux-native" than the Java-heavy Mesos stack.

Here is a unit file myapp.service for Fleet:

[Unit]
Description=My Web App
After=docker.service
Requires=docker.service

[Service]
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker kill web1
ExecStartPre=-/usr/bin/docker rm web1
ExecStartPre=/usr/bin/docker pull coolvds/webapp:v1
ExecStart=/usr/bin/docker run --name web1 -p 80:80 coolvds/webapp:v1
ExecStop=/usr/bin/docker stop web1

[X-Fleet]
Conflicts=myapp.service

The [X-Fleet] section is where the magic happens. Conflicts=myapp.service ensures that two instances of this container never land on the same host—instant high availability.

The "Boring" Option: Ansible & Docker CLI

Sometimes, boring is good. If you have fewer than 20 servers, you might not need an orchestrator. A simple Ansible playbook wrapping standard Docker commands is robust and easy to debug.

However, you run into the "port conflict" hell. Without an overlay network (which is still very experimental in Docker 1.0), you have to manage port mapping manually. -p 8080:80 on one app, -p 8081:80 on another. It gets messy fast.

Why Your Virtualization Choice Matters (The KVM Factor)

This is where things go wrong for most devs. I've seen teams trying to run Docker inside OpenVZ containers. Do not do this.

Pro Tip: Docker relies on kernel namespaces and cgroups. In OpenVZ, you share the kernel with the host node. If the host is running CentOS 6 (Kernel 2.6.32), Docker will either fail to start or run with severe instability because it expects Kernel 3.8+. You cannot upgrade the kernel inside an OpenVZ container.

This is why we strictly use KVM (Kernel-based Virtual Machine) at CoolVDS. With KVM, your VPS behaves like a dedicated server. You can install Ubuntu 14.04 LTS (released this April), which ships with Kernel 3.13—native, stable support for Docker 1.0. You get full isolation. If a neighbor's container goes rogue and panics their kernel, your instance stays up.

Performance: NVMe vs. The World

Containers are ephemeral. They spin up, write logs, and die. This generates a lot of random I/O. On standard SATA/SAS drives, we see "iowait" spike when launching 20 containers simultaneously.

We recently benchmarked disk I/O latency from our Oslo node against a standard cloud provider in Frankfurt.

Data Sovereignty in 2014

Post-Snowden, we are all looking at data locations differently. While the Safe Harbor agreement is still technically in effect, relying on US-hosted servers for Norwegian customer data is becoming a liability. The Norwegian Data Protection Authority (Datatilsynet) is becoming stricter about how personal data (Personopplysningsloven) is handled.

Hosting your container cluster on CoolVDS nodes in Oslo isn't just about millisecond latency to NIX (Norwegian Internet Exchange)—though that is nice. It is about knowing exactly where your physical drives are located.

Conclusion

If you are building a massive distributed system today, look at Mesos. If you want to experiment with the future, play with CoreOS Fleet. But whatever you choose, the foundation must be solid.

You need a Kernel 3.10+ capable OS, full KVM isolation, and storage that doesn't choke when you run docker-compose up. Don't let IOwait kill your application.

Ready to test Docker 1.0 in a production-ready environment? Spin up a KVM instance on CoolVDS in Oslo. It takes 55 seconds.