Why Your RAID Array Won't Save You

It is 3:00 AM. The pager goes off. Your primary database server just kernel panicked, and upon reboot, the filesystem is screaming about corrupted superblocks. You are running RAID 10, so you feel safe. You shouldn't.

RAID is redundancy. It is not a backup. RAID will happily mirror a corrupted file to both disks. RAID will instantaneously replicate an accidental rm -rf /var/lib/mysql across your entire array. If you do not have a snapshot-in-time backup living on a completely different physical machine, you do not have data. You have a ticking time bomb.

In the Nordic hosting market, where reliability is the currency we trade in, losing customer data is an extinction-level event. I have seen hosting providers in Oslo go under because they relied on local snapshots that vanished along with the SAN controller.

This guide is for the sysadmin who wants to sleep at night. We are going to build a bulletproof, automated backup system using standard tools available on any CentOS 6 or Ubuntu 10.04 LTS box. No expensive enterprise bloatware. Just raw, verifiable script logic.

The 3-2-1 Rule: Non-Negotiable

Before we touch a single config file, memorize this architecture. It is the only thing standing between you and professional ruin.

• 3 copies of your data.
• 2 different media types (e.g., Disk and Tape, or Local and Remote).
• 1 copy off-site (preferably in a different hazard zone).

For a VPS environment, "Tape" is metaphorical. We treat a separate storage server in a different datacenter as our second medium. If your primary server is in Oslo, your backup should ideally be in a different facility, perhaps Bergen or Stockholm, but ensure you respect the Personopplysningsloven (Personal Data Act). Data sovereignty matters.

Step 1: Database Consistency (The MySQL Headache)

Copying /var/lib/mysql while the database is running is a guarantee for corruption. You need a logical dump. If you are using MyISAM tables, you are forced to lock tables, which causes downtime. Stop living in 2005. Migrate to InnoDB.

With InnoDB, we can use the --single-transaction flag to get a consistent snapshot without locking the database for writes. This is critical for high-traffic e-commerce sites.

The Dump Script

Create a script at /usr/local/sbin/db_backup.sh:

#!/bin/bash

# Configuration
DB_USER="root"
DB_PASS="StrictlySecretPassword123!"
BACKUP_DIR="/backup/mysql"
DATE=$(date +"%Y-%m-%d_%H-%M")
RETENTION=7

# Ensure backup dir exists
mkdir -p $BACKUP_DIR

# Get list of databases, exclude system dbs
DATABASES=$(mysql -u $DB_USER -p$DB_PASS -e "SHOW DATABASES;" | grep -Ev "(Database|information_schema|performance_schema)")

for db in $DATABASES; do
    echo "Dumping $db..."
    mysqldump -u $DB_USER -p$DB_PASS --single-transaction --quick --lock-tables=false "$db" | gzip > "$BACKUP_DIR/$db-$DATE.sql.gz"
done

# Prune old backups
find $BACKUP_DIR -name "*.sql.gz" -mtime +$RETENTION -exec rm {} \;

echo "MySQL Backup Complete."

Make it executable: chmod +x /usr/local/sbin/db_backup.sh.

Pro Tip: Never store passwords in plain text if you can avoid it. Set up a .my.cnf in the user's home directory with restricted 600 permissions containing the credentials. This prevents the password from showing up in the process list (ps aux).

Step 2: Filesystem Sync with Rsync

rsync is the swiss-army knife of backups. It uses a delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination.

For a "Pull" configuration (where the backup server connects to the production server), you need SSH key authentication set up. This is safer than a "Push" config because if your production server gets compromised, the attacker cannot wipe your backups.

On your Backup Server (e.g., a storage-optimized instance):

rsync -avz --delete -e "ssh -p 22" root@production-server.coolvds.net:/var/www/html/ /backup/snapshots/production_www/

Breaking down the flags:

Step 3: Encryption and Security

Under the Norwegian Data Inspectorate (Datatilsynet) guidelines, handling personal data requires strict security controls. Storing unencrypted dumps on a remote server is negligent.

We can pipe our tar stream through GPG or OpenSSL before it hits the disk. Here is how to create an encrypted tarball on the fly:

tar -czf - /var/www/html | openssl enc -aes-256-cbc -salt -out /backup/www-backup.tar.gz.enc -pass pass:YourSuperSecretKey

This ensures that even if the physical disks of your backup server are stolen, the data remains inaccessible.

The Hardware Factor: Why I/O Matters

Running backups is I/O intensive. rsync has to walk the filesystem, reading metadata for millions of inodes. mysqldump performs heavy reads. On traditional spinning rust (HDD) VPS hosting, this often causes "I/O Wait" to spike, making your website sluggish for visitors during the backup window.

This is where infrastructure choice becomes architectural. I recently migrated a high-traffic Magento store to CoolVDS specifically for their SSD-accelerated storage tiers. The difference was night and day.

• Legacy HDD VPS: Backup took 4 hours. Site latency spiked to 2.5s.
• CoolVDS SSD VPS: Backup took 45 minutes. No noticeable latency impact.

When you are running KVM virtualization (which CoolVDS uses standard), you get true resource isolation. You are not fighting for disk time with a noisy neighbor who decided to compile a kernel on a shared OpenVZ container.

Step 4: Automation

Finally, tie it all together with cron. Edit your crontab with crontab -e:

# Run Database Backup at 2:00 AM
0 2 * * * /usr/local/sbin/db_backup.sh >> /var/log/db_backup.log 2>&1

# Run Off-site Sync at 3:00 AM (Pull from backup server)
# Note: This line goes on the BACKUP server, not the web server.
0 3 * * * /usr/local/sbin/sync_production.sh

Conclusion: Verify or Die

A backup you haven't restored is just a rumor. Dedicate one day a month to a "Fire Drill." Spin up a fresh instance on CoolVDS—it takes about 55 seconds—and attempt to restore your production environment from your backups. If it fails, fix the script. If it works, destroy the instance and sleep soundly.

Do not let a hardware failure become a business failure. Secure your data, respect the latency to Oslo, and choose a host that gives you the I/O throughput to handle disaster recovery without choking your production traffic.

Ready to upgrade your infrastructure? Deploy a high-performance SSD instance on CoolVDS today and stop worrying about I/O bottlenecks.