Backup Files to a Remote Server using Bash

Introduction

A great way to backup your Linux files is through automation. Linux contains many built in commands that can be used to automate this process. In this article, we will write a simple bash script to backup critical files to a remote host.

For this to work, a user account is needed on both hosts to transfer the files. The sending server will need the private SSH key and the receiving server will need the public SSH key. These keys are used to automatically authenticate to the remote server. Creating and deploying a SSH key pair for authentication is beyond the scope of this article.

The script will use the scp command as it uses SSH underneath the hood. This means it will natively automatically check for an identity file stored under the user’s account at ~/.ssh/config. When creating the SSH credentials, do not add a passphrase to the private key.

Secure copy (scp) command is good if you just want to copy a single file. You can use the -R switch to copy all files in a directory. If you need to backup multiple files, you can modify the script and just add additional variables, update checks, and then add more scp commands.

If you need to transfer entire directories consider using the rsync command. Rsync like scp should automatically use the ~/.ssh/conf file by default. However, rsync sometime has issues using an identity file, so do proper testing.

Finally, Consider setting up a CRON job to run the script nightly.

Procedure

Lets build a script on server1 and we will be connecting and coping our file to server2. First, set up the head of the script and call bash and add any comments.

#!/bin/bash
# backup.sh

Next, let’s set up the script variables.

DATE=`date`
DAY=$(date -d "1 day ago" +%d)
MONTH=$(date -d "1 day ago" +%b)
YEAR=$(date -d "1 day ago" +%Y)
SRC_DIR="/var/logs/$YEAR/$MONTH"
DST_DIR="/user1/backup/$YEAR/$MONTH"
FILE="important-file-$DAY.sql"

Next, lets run a check on the remote server and test if the file path exists!

# test if path exist on remote server.
# -d=test for file & if file exists is it a dir?
TEST=`ssh server2.company.com "test -d $DST_DIR && echo 1 || echo 0"`
if [ $TEST = 0 ] ; then
     echo "$DATE : $DST_DIR was not found. Making ..."
     # -p=make all missing directories in the file path.
     ssh server2.company.com "mkdir -p $DST_DIR"
else
     echo "$DATE : $DST_DIR was found."
fi

Finally, copy the file to the remote server.

# Copy file to remote server.
#-p=preserve permissions
scp -p $SRC_DIR/$FILE user1@server1:$DST_DIR

Lastly

Create a cronjob to run nightly at 9:00 PM.

References

https://unix.stackexchange.com/questions/127352/specify-identity-file-id-rsa-with-rsync