I’ve been conducting DR tests and site failovers for years using a myriad of host based and array based replication technologies.  By now the tasks of failing the host over from site A to site B and gaining access to replicated data is a highly predictable and controllable event.  What I often find is that little issues like time being out-of-sync due to NTP server issue, a host needing to be rejoined to the domain or the dreaded missing or fat fingered DNS entry tend to slow you down.

I recently ran a DR test where in the prior test a DNS entry was fat fingered, the bad DNS entry impacted the failback and extended the test time by about 5 hours.  Prior to this year’s test I decided to safeguard the DNS component of the test.  I crafted a small shell script to record and check the DNS entries (forward and reverse),  The plan would be as follows:

1. Capture DNS entries prior to the DR test and save as the production gold copy (known working production DNS records)
2. Capture DNS entries following the failover to the DR location and DNS updates.  Ensure that the DNS entries match the documented DR site IP schema.
3. Finally capture the DNS entries post failback to the production site.  Diff the pre-failover production site DNS entries (gold copy) with the post-failback production site DNS entries.

The fail-safe DNS checks proved to be very valuable, uncovering a few issues on failover and failback.  Below is my script, I ran the shell script from a Linux host, if you need to run on Windows and don’t want to rewrite you could try Cygwin (I don’t believe the “host” command is natively packaged with Cygwin but it could probably be compiled, haven’t looked around much)  or you could download VirtualBox and run a Linux VM. Hopefully you find this useful.

#!/bin/bash
#script to check dns registrations

if [ $# -lt 1 ]
then
  echo "Usage : $0 [prod|dr]"
  exit
fi

LOGFILE="checkdns_$1.log.`date +"%m%d%y-%H%M%S"`"
cp /dev/null $LOGFILE    # creates an empty $LOGFILE

echo "...Starting DNS record check"

while read DNSRECORD RECORDTYPES ; do
  for RECORDTYPE in $RECORDTYPES ; do
    echo "...Checking DNS [$RECORDTYPE] record for $DNSRECORD"
    host -t $RECORDTYPE $DNSRECORD | sort >>$LOGFILE
    sleep 1
  done
done < "hosts_$1.in"

echo "...Done"
echo "...Results logged to $LOGFILE"

Note:  you will need two input files:  “hosts_prod.in” and “hosts_dr.in”. These input files should contain your lookups for each site.

.in file example (syntax for .in files is “hostname | IP [space] record type”):
host1 a
host2 a
192.168.100.1 a
192.168.100.2 a

Syntax to execute the script is as follows “./checkdns.sh [prod | dr]”