Incremental email backups
I’ve had some interesting projects lately, most notably is an email backup script.
My first thoughts on this was to use RoboCopy and just have it run on log off. Unfortunately RoboCopy does not support incremental backups, thus meaning 10+ computers could be copying potentially 2GB+ files to the server every log-off; not good.
My attention was then steered towards Rsync, a tool which I have used before, that uses the ‘Delta Algorithm’ to scan for changes in a file and copy just those bits that have changed.
I toyed with the idea of having a Linux virtual machine running the server, though this most likely wouldn’t have worked, considering it would need considerable disk space and if something went wrong, well… Linux isn’t exactly the most user friendly of operating systems.
So then I remembered there was a Windows version of Rsync, though I had never actually managed to get it working, it seemed like the best place to start.
DeltaCopy is basically a wrapper for Rsync, running within a Cygwin environment. It’s open source, so no licensing costs, and it runs its own service with a quite small footprint.
My first itteration of DeltaCopy involved simply mounting the remote share and doing a local copy i.e. rsync.exe [source] [destination] which apparently is the wrong way to go about it. Every time a bit was updated, it took nearly 3 times longer than the whole file should’ve taken to copy. I discovered that if you use a share, basically you are downloading a copy of the file, scanning it locally, then sending the whole file back down the line.
The proper way of using it, apparently, is to have an rsync server where the backups are kept, and have the rsync client connect to the server. That way the server can scan for changes without having to send it down to the client. Then the server and client negotiate which bits need copying, then they do it.
After much frustration and many issues (mainly typos :P) I managed to get my test platform to copy from 1 machine to the other. I succeeded and managed to take the 3 minute transfer time down to 6 seconds. Impressive!
To implement the script I will need to have the rsync service installed on the server (which I have now done) and use Group Policy to deploy both the DeltaCopy client (particularly rsync.exe) and my script to run on log off (also done).
I have now come to the point where it should be working but unfortunately it keeps coming up with “Connection Timed Out”. My guess is the firewall on the server isn’t accepting the connections, so I’ll have to look into that and see what can be done.
I’ll need rsync’s port 873 opened up I believe. Whether that needs to be done using Group Policy or through the Firewall settings, I’m not sure.
Anyway, that’s it for the moment.
I think I’ll post my scripts up sometime, stay tuned.
Find them right here: Download backupScripts.zip 7.7KB
You will also need DeltaCopy from Synametrics Technology