Introduction
Running nightly file backups is a common task for administrators. How do we know the file was copied successfully with no errors? In this post, we will set up an ansible script and it will run a file integrity check using MD5 on both the source and the destination files to verify it was not corrupted during the copy process. In this process the Ansible server is assumed to be a separate server from both the source server and the designation server.
Specifically, we will tell Ansible to execute a bash script on the source and destination servers, gather the results and store them in a temp text file, then it will output the text file to the body of an email and send it to interested parties for review.
Create the Ansible Script
Add comments to the head of the script. I like to include an example of the command, so that it can be easily copied to the command line.
# Ansible script, validate.yml
# cmd: ansible-playbook -i inventory.ini validate.yml
Add the variables to the script. All ansible scripts start with three dashes. Also note the Ansible is very sensitive to the placement of the columns. The names, hosts, and tasks columns must be lined up exact or the script will not execute.
---
- name: Verify integrity using md5 checksums
hosts: server1.company.com:server2.company.com
gather_facts: False
Add the tasks that must be executed.
tasks:
- name: Check file integrity
script: /home/user1/validate.sh
ignore_errors: False
register: results
- name: Make a header for the results txt file.
shell: echo '<-- results -->'
register: title
delegate_to: localhost
run_once: true
- name: Create new results txt file.
local_action: copy content={{ title.stdout }}
dest=/home/user1/validate.txt
- name: Append results to txt file.
lineinfile:
dest: /home/user1/validate.txt
line: "{{ results.stdout }}"
insertafter: EOF
delegate_to: localhost
- name: get the date
shell: "date +%Y-%m-%d"
register: tstamp
delegate_to: localhost
Finally we will send an email to interested parties.
- name: Sending email
mail:
host: exchange.company.com
port: 25
from: hostname@company.com
to:
- user1@company.com
- user2@company.com
subject: Copied to backup server.
body: "Date: {{ tstamp.stdout }}\n\n
NOTE: The below results are for yesterday's files.\n\n
{{ lookup('file','/home/user1/validate.txt') }}"
delegate_to: localhost
run_once: True
Build the Bash Script
Remember, in Ansible, it will execute the code on all servers simultaneously. So, we don’t know what server’s results will be returned to Ansible first. That is why we need the server hostname.
Create the headers.
#!/bin/bash
# validate.sh
Create the variables.
date=`date -d yesterday +%Y-%m-%d`
host=`hostname | awk '{print tolower($0)}'`
day=`date -d yesterday +%d`
month=$(date -d yesterday +%b)
year=`date +%Y`
src_dir1="/var/ossec/logs/$year/$month"
dst_dir1="/mnt/storage/logs/$year/$month"
file="filename-$day.log"
Execute the comamnds, to gather the needed data.
# Get source file results
results1=$(ls -alhF "$src_dir1/$file")
md5sum1=$(md5sum "$src_dir1/$file" | awk '{ print $1 }' )
# Get destination file results
results2=$(ssh backupserver "ls -alhF $dst_dir1/$file")
md5sum2=$(ssh backupserver "md5sum $dst_dir1/$file" | awk '{ print $1 }')
Output the results. Remember these results will be returned to Ansible.
# output results, compare file file size and MD5 hash.
echo " $host File: $results1"
echo "Backup File: $results2"
echo ""
echo " $host MD5_Sum: $md5sum1"
echo "Backup MD5_Sum: $md5sum2"
This is my own method for verifying files were copied correctly. I hope you find it useful.