The Art of the Automated Bailout: Scripting Bulletproof Backups on Linux

There are two types of system administrators: those who have lost data, and those who are about to. I learned this the hard way in 2007, staring at a corrupted partition table on a Friday night while a client’s e-commerce site vanished into the digital void. RAID 5 didn't save me. The mirroring rebuilt the corruption perfectly. Since then, I don't trust hardware, I don't trust file systems, and I certainly don't trust manual intervention.

If you are managing a VPS in Norway or dedicated servers for European clients, your backup strategy is the only thing standing between you and a lawsuit. With the strict requirements of the Personopplysningsloven (Personal Data Act) enforced by Datatilsynet here in Norway, losing customer data isn't just an inconvenience—it's negligence.

The Golden Rule: 3-2-1

Before we touch a single config file, memorize this. Your data must exist in:

• 3 copies (Production + Local Backup + Remote Backup)
• 2 different media types (Disk + Tape/External Storage)
• 1 copy off-site (If the datacenter burns down, your data survives)

At CoolVDS, we see too many developers relying on snapshots alone. Snapshots are great for quick rollbacks before a yum update, but they are stored on the same storage array as your live data. If the SAN cracks, you lose everything.

The Weapon of Choice: Rsync & SSH Keys

Forget FTP. It's insecure and handles incremental changes poorly. We use rsync over SSH. It uses a delta-transfer algorithm, meaning if you change 1MB of a 10GB file, it only sends that 1MB. This is critical for keeping low latency on your production uplinks.

Step 1: Password-less Authentication

Your backup script needs to log in without human help. Generate a key pair on your production server (CentOS 5 / Debian Lenny example):

server1# ssh-keygen -t rsa -b 2048
# Hit enter for no passphrase
server1# ssh-copy-id -i ~/.ssh/id_rsa.pub backupuser@backup.coolvds.com

Now, test the connection. If you get a prompt, you failed. If you get a shell, you're ready to script.

The Script: Automating the Rotation

A backup script that fills up your disk is a time bomb. We need rotation. Here is a battle-tested Bash script I use on production servers. It keeps 7 daily backups and rotates them automatically.

#!/bin/bash
# /usr/local/sbin/backup_daily.sh

# Configuration
BACKUP_ROOT="/backup"
REMOTE_DEST="backupuser@10.0.0.50:/var/backups/remote/"
DATE=$(date +%F)
MYSQL_USER="root"
MYSQL_PASS="MySuperSecretPassword"

# 1. Database Dump (The Critical Part)
# We use --single-transaction for InnoDB to avoid locking tables.
# If you are still using MyISAM, expect table locks.
echo "Starting MySQL Dump..."
mkdir -p $BACKUP_ROOT/$DATE/mysql

for db in $(mysql -u $MYSQL_USER -p$MYSQL_PASS -e 'show databases' -s --skip-column-names); do
    mysqldump -u $MYSQL_USER -p$MYSQL_PASS --single-transaction --quick $db | gzip > $BACKUP_ROOT/$DATE/mysql/$db.sql.gz
done

# 2. File System Backup
echo "Archiving /var/www..."
tar -czf $BACKUP_ROOT/$DATE/files.tar.gz /var/www

# 3. Push to Remote Storage (Off-site)
# We use bandwidth limiting (--bwlimit) to avoid choking the network interface
echo "Syncing to remote..."
rsync -avz --bwlimit=2000 -e ssh $BACKUP_ROOT/$DATE $REMOTE_DEST

# 4. Cleanup old backups (older than 7 days)
find $BACKUP_ROOT -maxdepth 1 -type d -mtime +7 -exec rm -rf {} \;

echo "Backup Complete."

Pro Tip: Never rely on the exit code alone. Add a line to email the output of this script to your mobile device or BlackBerry. Silent failures are the enemy.

Database Integrity: MyISAM vs. InnoDB

If you are running a high-traffic forum or Magento store, you have likely moved to InnoDB. However, legacy PHP applications often default to MyISAM. The difference matters for backups.

At CoolVDS, our managed hosting templates are tuned for InnoDB by default in my.cnf (specifically innodb_buffer_pool_size), ensuring that your backups don't cause 502 Bad Gateway errors during peak hours.

The Hardware Factor: Why IOPS Matter

Running `tar` and `gzip` creates a massive spike in Disk I/O. If you are on a cheap VPS provider that oversells resources, this will cause "CPU Steal" time to skyrocket, making your web server sluggish. This is the "noisy neighbor" effect.

This is where infrastructure choice becomes strategic. You need storage capable of handling random writes without choking. While standard SATA drives spin at 7,200 RPM, we utilize enterprise-grade 15k RPM SAS drives and early-generation SSD storage arrays in RAID-10 configurations. This high-performance setup absorbs the I/O punishment of backup compression without degrading the performance of your Apache or Nginx processes.

Automating via Cron

The script is useless if it doesn't run. Add it to your crontab. Run it at 3:00 AM local time (CET), when Norwegian traffic is lowest.

# crontab -e
0 3 * * * /bin/bash /usr/local/sbin/backup_daily.sh >> /var/log/backup.log 2>&1

Summary

Automated backups are not about technology; they are about discipline. By combining `rsync`, proper MySQL dumping techniques, and reliable hardware, you insulate your business from disaster.

Don't wait for a hard drive failure to test your recovery plan. If you need a sandbox to test your restore scripts without risking production, deploy a CoolVDS instance. Our DDoS protection and high-speed internal network make it the ideal environment for a secure, secondary backup node.