Admin: New server guide

2026-04-14T10:02:36Z

New server guide

New page

= Setting Up Automated Backups with rsync =

This guide provides a comprehensive walkthrough for setting up automated backups for your server using the powerful `rsync` utility. We will cover creating robust `rsync` scripts, automating their execution with `cron`, and demonstrating how to transfer backups to a remote server for enhanced data safety. This is crucial for maintaining data integrity and ensuring business continuity in the event of hardware failure, accidental deletion, or security breaches.

== Prerequisites ==

Before you begin, ensure you have the following:

* A Linux server with root or sudo privileges. For reliable hosting with full root control, consider dedicated servers at [https://powervps.net/?from=32 PowerVPS].
* SSH access to your server.
* Basic understanding of the Linux command line.
* A separate server or storage location for your backups (the "backup server"). This can be another dedicated server, a VPS, or even a NAS device accessible via SSH.

== Understanding rsync ==

`rsync` is a versatile file-synchronization utility that efficiently transfers and synchronizes files between two locations. Its key advantages include:

* ''Delta-transfer algorithm'': `rsync` only transfers the *differences* between files, making it extremely efficient for subsequent backups.
* ''Preservation of permissions, ownership, timestamps'': It can maintain file metadata, crucial for restoring systems accurately.
* ''Compression'': Can compress data during transfer, saving bandwidth.
* ''Remote and local synchronization'': Works seamlessly between local directories and remote servers via SSH.

== Creating Your First rsync Backup Script ==

We'll start by creating a basic `rsync` script to back up a specific directory to another location on the same server. This serves as a foundational step before moving to remote backups.

=== Script Location ===

Create a directory to store your backup scripts:
<pre>
sudo mkdir -p /opt/scripts
sudo chown root:root /opt/scripts
sudo chmod 700 /opt/scripts
</pre>

=== The Backup Script ===

Now, create the `rsync` script file. We'll name it `backup_home.sh` for this example, assuming we want to back up the `/home` directory.

<pre>
sudo nano /opt/scripts/backup_home.sh
</pre>

Paste the following content into the file, replacing placeholders as needed:

<pre>
#!/bin/bash

# --- Configuration ---
SOURCE_DIR="/home/"
BACKUP_DIR="/mnt/backups/home/" # Ensure this directory exists and has sufficient space
LOG_FILE="/var/log/rsync_backup_home.log"
DATE=$(date +"%Y-%m-%d_%H-%M-%S")
REMOTE_USER="backupuser" # User on the remote backup server
REMOTE_HOST="your_backup_server_ip" # IP address or hostname of your backup server
REMOTE_DIR="/backups/server1/home/" # Directory on the remote server

# --- Options ---
# -a: archive mode (preserves permissions, ownership, timestamps, etc.)
# -v: verbose (shows files being transferred)
# -z: compress file data during the transfer
# --delete: delete extraneous files from dest dirs (use with caution!)
# --exclude: patterns to exclude from backup
# --log-file: write output to a log file
RSYNC_OPTIONS="-avz --delete --log-file=${LOG_FILE}"

# --- Pre-backup Checks ---
# Ensure source directory exists
if [ ! -d "$SOURCE_DIR" ]; then
echo "[$DATE] ERROR: Source directory '$SOURCE_DIR' does not exist. Aborting." | tee -a "$LOG_FILE"
exit 1
fi

# Ensure backup directory exists (for local testing or staging)
# For remote backups, this check is less critical as rsync will create it if needed on the remote side.
if [ ! -d "$BACKUP_DIR" ]; then
echo "[$DATE] WARNING: Local backup staging directory '$BACKUP_DIR' does not exist. Creating it." | tee -a "$LOG_FILE"
mkdir -p "$BACKUP_DIR"
if [ $? -ne 0 ]; then
echo "[$DATE] ERROR: Failed to create local backup staging directory '$BACKUP_DIR'. Aborting." | tee -a "$LOG_FILE"
exit 1
fi
fi

# --- Perform the Backup ---
echo "[$DATE] Starting backup of '$SOURCE_DIR' to remote '$REMOTE_USER@$REMOTE_HOST:$REMOTE_DIR'" | tee -a "$LOG_FILE"

rsync $RSYNC_OPTIONS \
--exclude='*.tmp' \
--exclude='cache/' \
"$SOURCE_DIR" "$REMOTE_USER@$REMOTE_HOST:$REMOTE_DIR"

# --- Post-backup Checks ---
RSYNC_EXIT_CODE=$?

if [ $RSYNC_EXIT_CODE -eq 0 ]; then
echo "[$DATE] Backup completed successfully." | tee -a "$LOG_FILE"
else
echo "[$DATE] ERROR: rsync exited with code $RSYNC_EXIT_CODE. Check log file for details." | tee -a "$LOG_FILE"
# Consider sending an email notification here for critical errors
fi

exit $RSYNC_EXIT_CODE
</pre>

* **`SOURCE_DIR`**: The directory you want to back up.
* **`BACKUP_DIR`**: A local staging directory. While we're primarily focusing on remote backups, having a local staging area can be useful for testing or if your remote connection is temporarily unavailable.
* **`LOG_FILE`**: Where `rsync` will write its output.
* **`REMOTE_USER`**: The username on your backup server. This user must have SSH access and write permissions to the `REMOTE_DIR`.
* **`REMOTE_HOST`**: The IP address or hostname of your backup server.
* **`REMOTE_DIR`**: The target directory on your backup server.
* **`RSYNC_OPTIONS`**:
* `-a` (archive): This is a crucial option that implies `-rlptgoD`. It recursively copies directories, preserves symbolic links, permissions, modification times, group, owner, and device files.
* `-v` (verbose): Provides detailed output of what `rsync` is doing. Helpful for debugging.
* `-z` (compress): Compresses file data during transfer. Saves bandwidth but uses more CPU.
* `--delete`: This option synchronizes the destination with the source. Any files in the destination that are no longer in the source will be deleted. **Use with extreme caution!** Ensure your `SOURCE_DIR` is correct before enabling this.
* `--exclude`: Allows you to skip specific files or directories. Useful for temporary files, caches, or large log files you don't need to back up.
* **Pre-backup Checks**: These ensure the script doesn't proceed if essential directories are missing, preventing potential errors.
* **Post-backup Checks**: The script checks the exit code of `rsync` to determine success or failure.

=== Making the Script Executable ===

Give your script execute permissions:
<pre>
sudo chmod +x /opt/scripts/backup_home.sh
</pre>

=== Testing the Script ===

Before automating, run the script manually to ensure it works as expected:
<pre>
sudo /opt/scripts/backup_home.sh
</pre>

Check the output and the log file (`/var/log/rsync_backup_home.log`) for any errors. Verify that files have been transferred to your `REMOTE_DIR` on the backup server.

== Setting Up SSH Key-Based Authentication ==

For `cron` to run `rsync` without prompting for a password, you need to set up SSH key-based authentication.

=== Generate SSH Key Pair ===

On your ''source'' server (the one you're backing up), generate an SSH key pair for the `root` user (or the user that will run the cron job):

<pre>
sudo su -
ssh-keygen -t rsa -b 4096
</pre>

Press Enter to accept the default file location (`/root/.ssh/id_rsa`) and leave the passphrase empty (press Enter twice). An empty passphrase is required for automated, non-interactive logins.

=== Copy Public Key to Backup Server ===

Now, copy the public key to your backup server. Replace `backupuser` and `your_backup_server_ip` with your actual details.

<pre>
ssh-copy-id backupuser@your_backup_server_ip
</pre>

You will be prompted for the `backupuser`'s password on the backup server. After this, you should be able to SSH from your source server to your backup server as `backupuser` without a password.

=== Test SSH Connection ===

Test the passwordless SSH connection:
<pre>
ssh backupuser@your_backup_server_ip
</pre>
You should log in directly without a password prompt. Type `exit` to return to your source server.

== Automating Backups with Cron ==

`cron` is a time-based job scheduler in Unix-like operating systems. We'll use it to run our `rsync` script automatically.

=== Edit the Crontab ===

Edit the crontab for the `root` user:
<pre>
sudo crontab -e
</pre>

If prompted, choose an editor (e.g., `nano`).

=== Add the Cron Job ===

Add the following line to the end of the file to run the backup script daily at 2:00 AM:

<pre>
0 2 * * * /opt/scripts/backup_home.sh > /dev/null 2>&1
</pre>

Let's break down the cron syntax:

* `0 2 * * *`: This specifies the schedule.
* `0`: Minute (0th minute of the hour)
* `2`: Hour (2 AM)
* `*`: Day of the month (every day)
* `*`: Month (every month)
* `*`: Day of the week (every day)
* `/opt/scripts/backup_home.sh`: The command to execute (your backup script).
* `> /dev/null 2>&1`: This redirects standard output and standard error to `/dev/null`. This prevents `cron` from emailing you the script's output every time it runs. The script itself logs to `/var/log/rsync_backup_home.log`.

=== Save and Exit ===

Save the crontab file and exit the editor. `cron` will automatically pick up the new job.

== Advanced Considerations and Best Practices ==

=== Incremental Backups with `--link-dest` ===

For more efficient storage and faster backups, you can use `rsync`'s `--link-dest` option. This creates hard links to unchanged files from a previous backup, effectively making each backup a full snapshot while only storing new or modified data.

Modify your script to include a `LATEST_BACKUP` variable pointing to the most recent backup directory and use `--link-dest`:

```bash
#!/bin/bash

# ... (previous configuration) ...

# --- New Configuration for Incremental Backups ---
INCREMENTAL_BASE="/backups/server1/home/" # Base directory for all backups on remote server
LATEST_LINK="${INCREMENTAL_BASE}latest" # A symbolic link to the most recent backup
CURRENT_BACKUP="${INCREMENTAL_BASE}backup_$(date +"%Y-%m-%d_%H-%M-%S")" # The new backup directory name

# --- Options ---
# --link-dest: use this for incremental backups
RSYNC_OPTIONS="-avz --delete --log-file=${LOG_FILE} --link-dest=${LATEST_LINK}"

# --- Pre-backup Checks ---
# ... (same as before) ...

# --- Perform the Backup ---
echo "[$DATE] Starting incremental backup of '$SOURCE_DIR' to remote '$REMOTE_USER@$REMOTE_HOST:$CURRENT_BACKUP'" | tee -a "$LOG_FILE"

rsync $RSYNC_OPTIONS \
--exclude='*.tmp' \
--exclude='cache/' \
"$SOURCE_DIR" "$REMOTE_USER@$REMOTE_HOST:$CURRENT_BACKUP"

RSYNC_EXIT_CODE=$?

if [ $RSYNC_EXIT_CODE -eq 0 ]; then
echo "[$DATE] Backup completed successfully. Updating latest link." | tee -a "$LOG_FILE"
# Update the 'latest' symbolic link on the remote server
ssh "$REMOTE_USER@$REMOTE_HOST" "rm -f ${INCREMENTAL_BASE}latest && ln -s ${CURRENT_BACKUP} ${INCREMENTAL_BASE}latest"
if [ $? -ne 0 ]; then
echo "[$DATE] WARNING: Failed to update 'latest' symbolic link on remote server." | tee -a "$LOG_FILE"
fi
else
echo "[$DATE] ERROR: rsync exited with code $RSYNC_EXIT_CODE. Check log file for details." | tee -a "$LOG_FILE"
fi

exit $RSYNC_EXIT_CODE
```

* **`INCREMENTAL_BASE`**: The root directory on the backup server where all dated backup directories will reside.
* **`LATEST_LINK`**: A symbolic link that always points to the most recent successful backup. `rsync` uses this to find unchanged files.
* **`CURRENT_BACKUP`**: The name of the new directory for the current backup.
* **`--link-dest=${LATEST_LINK}`**: This tells `rsync` to look at the directory pointed to by `LATEST_LINK` for unchanged files. If a file hasn't changed, it creates a hard link to it in the `CURRENT_BACKUP` directory instead of copying it again.
* **Updating `latest` symlink**: After a successful backup, we use `ssh` to remotely update the `latest` symbolic link to point to the newly created backup directory. This is critical for the next incremental backup.

=== Backup Rotation and Pruning ===

Incremental backups can consume significant disk space over time. Implement a strategy to remove old backups. You can do this with a separate script that runs after your `rsync` job, or as part of the `rsync` script itself.

Example script to remove backups older than 30 days (run this *after* your main backup script):

```bash
#!/bin/bash

BACKUP_ROOT="/backups/server1/home/" # Base directory on the remote server
DAYS_TO_KEEP=30

echo "Pruning backups older than $DAYS_TO_KEEP days in $BACKUP_ROOT..."
ssh backupuser@your_backup_server_ip "find ${BACKUP_ROOT} -maxdepth 1 -type d -name 'backup_*' -mtime +${DAYS_TO_KEEP} -exec echo 'Deleting old backup: {}' \; -exec rm -rf {} \;"
echo "Pruning complete."
```

Add this script to your `crontab` to run after the main backup job.

=== Encrypting Backups ===

For sensitive data, consider encrypting your backups. Tools like `gpg` can be used. You would typically encrypt the data *before* sending it to the backup server, or encrypt the backup archive.

=== Monitoring and Alerting ===

* **Log Rotation**: Use `logrotate` to manage your `rsync` log files to prevent them from growing indefinitely.
* **Email Notifications**: Enhance your script to send email alerts on backup failures. You can use the `mail` command for this.
* **Disk Space Monitoring**: Regularly monitor the disk space on your backup server.

== Troubleshooting Common Issues ==

* **''Permission Denied'' when running script**:
* Ensure the script has execute permissions (`chmod +x`).
* Verify the user running the script (e.g., `root` via `sudo crontab -e`) has read access to source directories and write access to log/backup directories.
* **''Password prompt'' during `rsync` execution**:
* SSH key-based authentication is not set up correctly. Revisit the "Setting Up SSH Key-Based Authentication" section.
* Ensure the `ssh-copy-id` command was used successfully and test the passwordless SSH connection manually.
* Check permissions on `~/.ssh/` and `~/.ssh/authorized_keys` on the backup server. They should be `700` and `600` respectively for the user running the backup.
* **''rsync: command not found''**:
* Ensure `rsync` is installed on your server: `sudo apt update && sudo apt install rsync` (Debian/Ubuntu) or `sudo yum install rsync` (CentOS/RHEL).
* **''rsync exited with code 23'' (Partial transfer due to vanished source files)**:
* This usually means some files were deleted or moved on the source *during* the `rsync` process. This is generally not a critical error if the backup is otherwise complete, but it indicates that the source data changed mid-backup.
* **''rsync exited with code 24'' (Partial transfer due to vanished destination files)**:
* This is more serious. It can indicate issues with the destination filesystem or permissions.
* **''rsync exited with code 126'' (Command invoked cannot execute)**:
* The script file does not have execute permissions.
* **''rsync exited with code

[[Category:Backup]]

Setting Up Automated Backups with rsync - Revision history

Admin: New server guide