Console Login

Disaster Recovery in the Shadow of GDPR: A DevOps Survival Guide for 2018

Disaster Recovery in the Shadow of GDPR: A DevOps Survival Guide for 2018

It is May 21, 2018. If you work in IT in Europe, you are likely tired of hearing the acronym "GDPR." The enforcement date is four days away. But while your legal team is panicking about cookie consent banners, we—the system architects and sysadmins—face a much darker reality. If a server melts down and data is permanently lost, we aren't just dealing with downtime anymore. We are dealing with Article 32 violations and potential reporting to Datatilsynet (The Norwegian Data Protection Authority).

I have seen it happen. A junior dev runs a malformed migration script on production at 4 PM on a Friday. The database locks up, the disk fills with binary logs, and suddenly your e-commerce platform is serving 500 errors to customers in Oslo. If you don't have a recovery plan that executes in minutes, you are bleeding money. If you lose that customer data entirely, you might be bleeding your company's existence.

This guide isn't about buying expensive enterprise backup appliances. It is about using standard Linux tools, robust KVM virtualization (like we use at CoolVDS), and common sense to build a bulletproof Disaster Recovery (DR) plan.

The "3-2-1" Rule is Not Optional

You know the drill, but are you actually doing it? Three copies of data, two different media, one offsite. In the context of VPS hosting, "media" usually means different storage backends.

Pro Tip: Snapshots are NOT backups. Snapshots are crash-consistent states stored on the same storage array as your live disk. If the SAN dies, your snapshot dies with it. Treat snapshots as a convenience, not a DR strategy.

Database Consistency: The Hard Truth

File-level backups of a running database are useless. If you are just rsyncing `/var/lib/mysql` while mysqld is running, you are backing up corruption. You need transactional consistency.

For MySQL 5.7 (standard in Ubuntu 16.04 and 18.04), you must ensure your binary logs are safe. In your `my.cnf`, strict ACID compliance is the only way to minimize data loss during a crash.

[mysqld]
# Ensure durability of ACID transactions
innodb_flush_log_at_trx_commit = 1
sync_binlog = 1

# GTID for easier failover and replication
gtid_mode = ON
enforce_gtid_consistency = ON

# Binary logging settings
log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 7

Setting innodb_flush_log_at_trx_commit = 1 forces a write to disk on every transaction commit. Yes, on spinning rust (HDD), this kills performance. This is why NVMe storage is critical. On CoolVDS NVMe instances, the I/O latency penalty is negligible, giving you safety without the speed tax.

Automating the "Hot" Backup

We need a script that runs nightly (or hourly), locks the tables for the minimum amount of time, dumps the data, compresses it, and ships it off-server. Here is a production-ready script utilizing mysqldump with the --single-transaction flag to avoid locking InnoDB tables.

#!/bin/bash

# Configuration
DB_USER="backup_user"
DB_PASS="StrongPassword123!"
BACKUP_DIR="/mnt/backups/mysql"
DATE=$(date +%Y-%m-%d_%H%M)
RETENTION_DAYS=7
REMOTE_HOST="user@offsite-storage.example.com"
REMOTE_DIR="/home/user/backups/"

# Ensure backup directory exists
mkdir -p $BACKUP_DIR

# 1. Dump all databases
# --single-transaction: crucial for InnoDB to avoid locking
# --routines --triggers: don't forget stored procedures
/usr/bin/mysqldump -u$DB_USER -p$DB_PASS --all-databases \
  --single-transaction --routines --triggers --events \
  | gzip -9 > "$BACKUP_DIR/full_backup_$DATE.sql.gz"

# 2. Verify the file size is not zero (basic check)
FILESIZE=$(stat -c%s "$BACKUP_DIR/full_backup_$DATE.sql.gz")
if [ $FILESIZE -lt 1024 ]; then
    echo "Backup failed! File too small." | mail -s "Backup Alert" admin@example.com
    exit 1
fi

# 3. Ship it offsite securely
scp -i /root/.ssh/id_rsa_backup "$BACKUP_DIR/full_backup_$DATE.sql.gz" $REMOTE_HOST:$REMOTE_DIR

# 4. Cleanup old local backups
find $BACKUP_DIR -name "*.sql.gz" -mtime +$RETENTION_DAYS -delete

echo "Backup $DATE completed successfully."

Make sure this script is executable: chmod +x /usr/local/bin/backup_mysql.sh. Then add it to your crontab.

File Synchronization: The `rsync` Approach

Your database is safe, but what about your `nginx` configs, your SSL certificates (Let's Encrypt), and your application code? If you are not using a CI/CD pipeline that redeploys from Git, you need to back up the filesystem.

Using `rsync` is standard, but you must exclude the junk. You don't want to back up `/proc`, `/sys`, or temporary cache files.

rsync -avz -e "ssh -p 22" \
  --exclude 'cache/' \
  --exclude 'logs/' \
  --exclude 'tmp/' \
  /var/www/html/ \
  backup_user@remote-storage:/var/backups/web/

Small flags matter. The -z flag compresses data during transfer, which is vital if you are pushing data from a server in Oslo to a backup node in Frankfurt to minimize bandwidth costs.

The Recovery Drill (The Part Everyone Skips)

A backup is Schrödinger's file: it both exists and doesn't exist until you try to restore it. You must simulate a disaster. Spin up a fresh CoolVDS instance. It takes about 55 seconds to provision. Once it is up, attempt to restore your service from your backups.

  1. Install dependencies: apt-get install nginx mariadb-server
  2. Restore DB: zcat full_backup_2018-05-21.sql.gz | mysql -u root -p
  3. Restore Files: Rsync the web files back to /var/www/html.
  4. DNS Switch: Lower your TTL (Time To Live) on your DNS records to 300 seconds (5 minutes) before a migration or major update.

Comparison: Manual Recovery vs. Automated Failover

Feature Manual Restore (Cold) Master-Slave Replication (Warm)
RTO (Recovery Time Objective) 1 - 4 Hours < 5 Minutes
RPO (Recovery Point Objective) Last Night's Backup Real-time (milliseconds)
Cost Low (Storage only) Double (2x Compute Instances)
Complexity Low High (Requires Monitor/Failover logic)

The CoolVDS Advantage in 2018

In 2018, the bottleneck for recovery is almost always Disk I/O. Restoring a 50GB database dump on a standard SATA VPS can take hours due to IOPS limits. This is where hardware selection becomes part of your DR strategy.

We use KVM (Kernel-based Virtual Machine) because it offers true hardware isolation. Unlike OpenVZ containers, which share a kernel and often suffer from "noisy neighbors," a KVM slice on CoolVDS guarantees your resources. When you are gunning gzip -d to decompress a massive backup during a crisis, you need every CPU cycle you paid for.

Furthermore, regarding GDPR: knowing exactly where your data sits is paramount. CoolVDS infrastructure is strictly governed, ensuring your data remains within the expected legal jurisdictions, a critical requirement for Norwegian businesses handling EU citizen data.

Final Checklist for May 25th

  • Encrypt your backups at rest. Use gpg or similar tools.
  • Test your restore process on a clean node.
  • Document the process. If you are hit by a bus, can your junior admin restore the system?

Don't wait for the hardware to fail or the hackers to strike. Disaster recovery is about confidence. Deploy a sandbox instance on CoolVDS today and break things so you know how to fix them when it counts.