Subversion incremental backup
Par Pierrick, mardi 17 avril 2007 à 11:48 / categorie: GNU/linux / tags: / #98 / rss

When your Subversion repository gets bigger and bigger, you need to find a solution to backup only what's new, and not the whole repository. Thanks to Subversion revisions, we can easily identify what's new since last backup. I've used this principle to write a Perl script making incremental backup.
Note: tos means Talend Open Studio. It is the Subversion repository I need to backup.
#!/usr/bin/perl use strict; use warnings; my $tos_repos = '/home/fbonan/svn/tos'; my $savedir = '/home/fbonan/save/tos'; my $last_saved_file = $savedir.'/last_saved.txt'; # read the last saved revision from the file open(LAST_SAVED, '<', $last_saved_file); my $last_saved = <LAST_SAVED>; chomp $last_saved; close(LAST_SAVED); # get informed of the current last revision (head) my $head = `svnlook youngest $tos_repos`; chomp $head; # of course, if the head is not younger than the last saved revision # it's useless to go on backuping if ($last_saved == $head) { exit(); } # if the last saved is 1000 and the head is 1023, we want the backup # from 1001 to 1023 my $from = $last_saved + 1; my $to = $head; # the backup filename looks like tos-01001_01023.svndump my $dumpfile = sprintf( $savedir.'/tos-%05u_%05u.svndump', $from, $to ); my $command = sprintf( 'svnadmin dump -q -r%u:%u --incremental %s > %s', $from, $to, $tos_repos, $dumpfile ); system($command); # here is the most interesting part of this script, how to make sure # the dump is a success? I've chosen to read its content to find the # information that the head revision was saved. If yes, I consider the dump # backup as a success. It sounds quite reliable. if (grep /^Revision-number: $to/, `grep --text ^Revision-number: $dumpfile`) { open(LAST_SAVED, '>', $last_saved_file); print LAST_SAVED $to, "\n"; close(LAST_SAVED); # here we compress the dump file system('gzip '.$dumpfile); # let's add the md5sum of the file to MD5SUMS file storing md5sums of # all Subversion backups chdir($savedir); use File::Basename; system('md5sum '.basename($dumpfile).'.gz >> '.$savedir.'/MD5SUMS'); }
At each backup session, the script stores the information "until which revision the last dump backuped?" in a file last_saved.txt. The interesting idea of this script is the way it checks the success of the backup. I needed this check because my remote provider sometimes kills the process. To check the dump, I check that the last revision I wanted to save is indeed saved in the dump. I do this by grepping the dump content.
This script needs to have at least one previous backup. The first one must be done by hand.
$ svnadmin dump -r0:9 --incremental /home/fbonan/svn/tos > tos-00000_00009.svndump * Dumped revision 0. * Dumped revision 1. * Dumped revision 2. * Dumped revision 3. * Dumped revision 4. * Dumped revision 5. * Dumped revision 6. * Dumped revision 7. * Dumped revision 8. * Dumped revision 9. $ gzip tos-00000_00009.svndump $ echo 9 > last_saved.txt
Add the incremental backup script in your crontab (crontab -e)
$ crontab -l # m h dom mon dow command # every three hours, incremental backup of TOS repository 0 */3 * * * /home/fbonan/script/tos_svndump.pl
After a few days, your backup directory will look like:
$ ls -1 MD5SUMS last_saved.txt tos-00000_00009.svndump.gz tos-00010_01999.svndump.gz tos-02000_02399.svndump.gz [...] tos-02783_02787.svndump.gz tos-02788_02792.svndump.gz tos-02793_02801.svndump tos-02793_02809.svndump.gz tos-02810_02818.svndump.gz $ cat last_saved.txt 2818
Here you see that there was a problem during backup of revision 2793 to 2801. 3 hours later (due to the crontab configuration), backup script runs once more and saves 2793 to 2809.
Commentaires
Aucun commentaire pour le moment.
Ajouter un commentaire
Les commentaires pour ce billet sont fermés.