Disaster Recovery in the GDPR Era: Surviving the 3 AM Catastrophe

There is a specific kind of silence that occurs at 3:14 AM when your monitoring dashboard turns red. It’s not peaceful. It’s the sound of revenue vanishing and, since May 25th of this year, the potential sound of the Norwegian Data Protection Authority (Datatilsynet) knocking on your door. If you are running mission-critical workloads in 2018 without a tested Disaster Recovery (DR) plan, you are not a systems administrator; you are a gambler.

I have spent the last decade debugging production environments across Europe. I have seen RAID controllers lie about write commits. I have seen rm -rf executed on the wrong terminal window. Hardware fails. Humans err. The only constant is entropy.

This guide isn't about buying expensive enterprise backup software. It is about using standard, battle-hardened Linux tools available on Ubuntu 18.04 and CentOS 7 to ensure that when the inevitable happens, you can restore your services before your morning coffee gets cold. We will focus on data integrity, Recovery Time Objectives (RTO), and keeping your data safely within Norwegian borders.

The RPO and RTO Reality Check

Before we touch a single config file, you need to define two metrics. If you don't know them, you can't design a solution.

• Recovery Point Objective (RPO): How much data can you afford to lose? One hour? One transaction?
• Recovery Time Objective (RTO): How long can you be offline?

For most SMEs hosting on a standard VPS in Norway, an RPO of 24 hours is unacceptable. We aim for 15 minutes. However, low RTO requires high I/O throughput. This is where the underlying infrastructure matters. Restoring 500GB of data on spinning rust (HDD) takes hours. On NVMe storage—which is standard on CoolVDS instances—it takes minutes. You cannot script your way out of slow hardware.

Database Consistency: The Silent Killer

A file copy of a running database is useless. If you just cp -r /var/lib/mysql while the daemon is writing, you get corrupted tables. You need a consistent snapshot.

For MySQL 5.7 or MariaDB 10.2 (common stacks in 2018), mysqldump is reliable, but it locks tables if you aren't careful. For InnoDB tables, the --single-transaction flag is mandatory. It ensures data consistency without stopping the world.

The Robust Dump Script

Do not rely on ad-hoc commands. Use a script like this, which rotates backups and ensures compression.

#!/bin/bash

# Configuration
DB_USER="backup_user"
DB_PASS="ComplexPassword123!"
BACKUP_DIR="/var/backups/mysql"
DATE=$(date +%Y-%m-%d_%H-%M)
RETENTION_DAYS=7

# Ensure backup directory exists
mkdir -p $BACKUP_DIR

# Dump all databases
# --single-transaction: Does not lock InnoDB tables
# --quick: Retrieves rows row-by-row rather than retrieving the whole result set
echo "Starting backup for $DATE..."

mysqldump -u$DB_USER -p$DB_PASS --all-databases --single-transaction --quick --events --routines --triggers | gzip > "$BACKUP_DIR/db_backup_$DATE.sql.gz"

# Check if the pipe command succeeded
if [ ${PIPESTATUS[0]} -ne 0 ]; then
    echo "FATAL: Backup failed"
    exit 1
fi

# Cleanup old backups
find $BACKUP_DIR -name "db_backup_*.sql.gz" -mtime +$RETENTION_DAYS -delete

echo "Backup $DATE complete and compressed."

Make this executable with chmod +x and test it immediately.

Filesystem Replication: rsync vs. ZFS

For static files (uploads, configuration), rsync is the universal soldier. It is efficient because it only transfers deltas. However, if you are moving data between a production server in Oslo and a backup node in, say, Trondheim (to keep data in Norway/EEA for GDPR compliance), encryption is non-negotiable.

Standard Rsync Over SSH

The following command synchronizes your web root to a remote server. It preserves permissions, ownership, and times.

rsync -avz -e ssh /var/www/html/ remote_user@backup.coolvds.net:/var/backups/web_root/

But let's talk about the superior option: ZFS. If your VPS provider supports custom kernels or offers ZFS-backed storage (like CoolVDS does for specific storage tiers), you can utilize snapshots. ZFS snapshots are atomic and instantaneous.

Creating a snapshot takes zero seconds:

zfs snapshot zpool/data@backup_2018-08-06

Sending that snapshot to a remote server is where the magic happens. It sends the block-level differences, which is significantly faster than rsync scanning millions of small PHP files.

# Send a snapshot stream over SSH to a remote ZFS pool
zfs send zpool/data@backup_2018-08-06 | ssh root@remote-backup-server "zfs recv backup_pool/data"

Pro Tip: Never store backups on the same physical disk as the OS. In a virtualized environment, ensure your backup volume is mounted separately. CoolVDS allows you to attach block storage volumes that persist even if the root instance is terminated. Use them.

Automating the Routine

Manual backups are failed backups. We use cron. Open your crontab with crontab -e and add the schedule. Run your database dumps during off-peak hours (usually 03:00 or 04:00 Oslo time).

# m h  dom mon dow   command
30 3 * * * /usr/local/bin/mysql_backup.sh >> /var/log/mysql_backup.log 2>&1
0 4 * * * /usr/bin/rsync -avz -e ssh /var/www/ user@backup_server:/backups/ >> /var/log/rsync.log 2>&1

The "Maintenance Mode" Failover

During a disaster, you don't want users seeing PHP errors or timeout screens. You want a polite "Maintenance" page. I keep a lightweight Nginx configuration ready to swap in. This config serves a static HTML file and returns a 503 status code (so Google doesn't de-index you).

server {
    listen 80;
    server_name example.no;

    root /var/www/maintenance;
    index index.html;

    location / {
        return 503;
    }

    error_page 503 @maintenance;
    location @maintenance {
        rewrite ^(.*)$ /index.html break;
    }
}

Validating the Archives

A backup is Schrödinger's file until you restore it. I recommend a monthly "Fire Drill." Spin up a fresh KVM instance on CoolVDS—since they bill hourly, this costs pennies—and attempt a full restore.

1. Deploy fresh instance.
2. Install LEMP stack.
3. Import database.
4. Verify application loads.

If you encounter a `checksum mismatch` or a corrupted gzip file during this drill, you just saved your company. If you find this out during a real emergency, update your LinkedIn profile.

Comparison: Backup Strategies

Why Infrastructure Choice is a DR Decision

We often treat hosting as a commodity, but in 2018, the gap between legacy VPS and modern infrastructure is widening. When a disaster strikes, your bottleneck is almost always I/O.

CoolVDS is built on a pure NVMe architecture. When you are untarring a 50GB backup file, the difference between a SATA SSD (common in the industry) and NVMe is not just percentage points; it is orders of magnitude. Furthermore, CoolVDS leverages KVM (Kernel-based Virtual Machine) virtualization. Unlike OpenVZ, KVM provides true hardware isolation. If a neighbor abuses their disk, your restore process doesn't stall.

Additionally, for those of us operating under the strict new GDPR mandates, CoolVDS guarantees data residency in Norway. Your backups don't accidentally drift to a data center in Virginia.

Final Thoughts

Disaster recovery is boring until it is vital. The scripts above are simple, but they work. They rely on tools that have existed for years because stability beats novelty when your hair is on fire.

Don't wait for the hardware to fail. It will. Check your disk health today:

sudo smartctl -H /dev/vda

If that result makes you nervous, or if your current host takes 12 hours to respond to a ticket, it is time to move. Deploy a high-availability NVMe instance on CoolVDS and sleep through the night.