Docker Orchestration in Late 2014: Fig, Fleet, or Madness?

Let’s be honest with ourselves. We all fell in love with Docker back in version 0.8. It solved the "works on my machine" problem that has plagued developers since the dawn of time. But now that we are sitting on Docker 1.2 (released just last month), we are facing a new, much uglier beast: How do you actually run this thing in production across multiple servers without losing your mind?

If you are still SSH-ing into five different boxes and manually typing docker run commands, you are doing it wrong. I’ve seen production environments melt down because a sysadmin forgot which port was mapped to the host on Server B. We need orchestration, but the tools market right now is fragmented, chaotic, and frankly, full of alpha-quality scripts masquerading as enterprise software.

Today, we aren't looking at marketing fluff. We are looking at the actual tools available right now, in September 2014, to manage container sprawl. We’ll look at Fig for composition, CoreOS Fleet for distributed init, and briefly touch on that new project from Google everyone is whispering about.

The "Link" Problem

Before we look at the tools, understand the pain point. Docker's --link flag works great on a single host. You link your web container to your database container, and magic happens in /etc/hosts.

# The old manual way
$ docker run -d --name db training/postgres
$ docker run -d -P --name web --link db:db training/webapp python app.py

But the second you need to scale horizontally—putting the database on a high-IOPS storage node and the web heads on cheap compute nodes—--link breaks. It doesn't span hosts. This is where orchestration comes in.

Contender 1: Fig (The Developer's Darling)

If you aren't using Fig yet, stop reading and install it. It is hands down the best way to define multi-container environments. Instead of a bash script full of docker commands, you get a clean YAML file.

Here is a snippet from a fig.yml I used last week for a Python project:

web:
  build: .
  command: python app.py
  ports:
   - "5000:5000"
  volumes:
   - .:/code
  links:
   - redis
redis:
  image: redis

Then you just run:

$ fig up

The Verdict: Fig is brilliant for development environments and single-host deployments. But for multi-host production? It’s not there yet. It doesn't natively handle cross-host networking without serious hacks involving Ambassador containers.

Contender 2: CoreOS & Fleet (The Cluster Approach)

For those of us managing serious infrastructure—think high-availability clusters serving traffic to Oslo and Stockholm—CoreOS is the current heavyweight champion. They stripped Linux down to the bare essentials and replaced SysVinit/Upstart with systemd and fleet.

Fleet essentially treats your entire cluster as one giant init system. You submit a unit file, and fleet decides where to run it. Here is a production example of a systemd unit file for a high-availability Nginx container:

[Unit]
Description=My Web App
After=docker.service
Requires=docker.service

[Service]
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker kill webapp1
ExecStartPre=-/usr/bin/docker rm webapp1
ExecStartPre=/usr/bin/docker pull coolvds/webapp:latest
ExecStart=/usr/bin/docker run --name webapp1 -p 80:80 coolvds/webapp:latest
ExecStop=/usr/bin/docker stop webapp1

[X-Fleet]
Conflicts=webapp*.service

Note the [X-Fleet] section. The Conflicts directive ensures that two instances of this web app never land on the same physical host. This provides instant high availability. If one node in your cluster goes down, fleet reschedules the container elsewhere.

Pro Tip: When using CoreOS on CoolVDS, configure your cloud-config to use the private networking interface for etcd peer communication. If you run etcd over the public internet, you are adding latency and inviting security headaches. Keep your consensus protocol traffic local.

The Wildcard: Kubernetes

Google open-sourced Kubernetes back in June. I've compiled the binaries and played with it. The concepts—Pods, Replication Controllers, Services—are revolutionary. They solve the networking issue by giving every pod its own IP address.

However, as of right now (v0.x), it is not ready for production. The setup is complex, requiring a SaltStack minion setup or massive manual configuration. Keep an eye on it for 2015, but don't bet your CTO's salary on it today.

The Foundation: Why Hypervisors Still Matter

There is a dangerous misconception spreading that because Docker is lightweight, the underlying VPS doesn't matter. "Just throw it on the cheapest cloud," they say. This is negligent.

Docker shares the host kernel. If you are on a budget provider using OpenVZ (container-based virtualization), you are effectively running containers inside a container. You cannot load your own kernel modules. You are stuck with whatever ancient 2.6.32 kernel the host is running. Docker screams for Kernel 3.8+.

This is why we standardized on KVM at CoolVDS. KVM gives you a dedicated kernel. You can install Ubuntu 14.04 LTS, get the 3.13 kernel, and run Docker the way it was intended. Furthermore, our I/O isolation ensures that a "noisy neighbor" thrashing their disk doesn't starve your database container of IOPS.

Performance & Data Sovereignty

If you are deploying for Norwegian clients, latency and law are your two constraints.

1. Latency: Running your cluster in US-East when your customers are in Trondheim adds 100ms+ latency. TCP handshakes take longer. SSL negotiation takes longer. Hosting locally in Norway or Northern Europe keeps that ping under 30ms.
2. Data Sovereignty: Under the Norwegian Personal Data Act (Personopplysningsloven), you are responsible for where your user data lives. While "Safe Harbor" currently exists for US transfers, the legal ground is shaking. Keeping data on servers physically located in Europe—specifically on a provider like CoolVDS that respects these boundaries—is the only risk-free move.

Summary: What to Use Today?

If you are building a complex stack today, here is the battle-tested path:

• Development: Use Fig. It's clean and keeps your developers sane.
• Production: Use CoreOS + Fleet. It handles scheduling and HA better than manual scripts.
• Infrastructure: Host it on KVM-based instances with high-speed SSDs (or NVMe if you can find it—we have it). Do not trust your production data to shared kernels.

The container ecosystem is moving fast. Docker 1.3 is rumored to bring digital signatures, which will be huge for security. But until then, build on rock-solid infrastructure.

Ready to build your cluster? Deploy a CoreOS-ready KVM instance on CoolVDS in under 55 seconds and stop fighting with OpenVZ legacy kernels.