BACKUP & SYNC YOUR MUSIC

Like many of you, I have a digital music collection consisting of mp3 files. My collection started with files I ripped from CDs I own. I predominantly now add to my collection with music I purchase online. Since I pay good money for my music, it is important to me that I do not lose any of the music I downloaded. My collection today takes up approximately 25gb of space. This article describes my strategy for maintaining and managing my music collection.

I store my music collection on a Linux desktop that runs Samba and NFS, allowing my music to be listened to by Windows machines as well as the Linux and Unix computers I have at home. My normal, personal computer is a laptop running Ubuntu 8.04. When at home, I access the music server via NFS and run Rhythmbox to listen to my collection. However, I sometimes need to travel; having my music collection available to me is important. So, I purchased an inexpensive 160gb USB-powered portable disk drive ($99 US), which provides both a backup for my music collection and the ability to listen to music when I'm away from home.

There are so many ways of creating a backup of a set of files. Many articles have already been published, and many more will be published. I cannot promise that my solution is the “best”. However, it does serve my needs, and it may therefore serve your's.

As I mentioned, I frequently add music to my collection, and I sometimes delete individual pieces of music. I did not want a backup solution that would simply add the new files to some backup directory. If I deleted a song, I also wanted that song removed from my backup. I wanted a simple mechanism that would maintain an exact duplicate of my current collection (namely removing the deleted pieces and adding the new music). I also didn't want it to run for hours (remember, I have 25gb of music). The solution I chose was to use the program rdiff-backup. The software and documentation can be found at the rdiff-backup home page, http://www.nongnu.org/rdiff-backup/. Ubuntu's Synaptic Package Manager makes installing rdiff-backup a pleasure. Do a search for rdiff-backup, and install the product when it appears (note: it also requires installation of Python).

The best description of rdiff-backup comes from the author of the software itself:

“rdiff-backup backs up one directory to another, possibly over a network. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves subdirectories, hard links, dev files, permissions, uid/gid ownership, modification times, extended attributes, acls, and resource forks. Also, rdiff-backup can operate in a bandwidth efficient manner over a pipe, like rsync. Thus, you can use rdiff-backup and ssh to securely back a hard drive up to a remote location, and only the differences will be transmitted. Finally, rdiff-backup is easy to use and settings have sensible defaults.

”

To automate the execution of rdiff-backup, I created a script (Fig.1).

Line #2 defines an output file to store the ongoing day-to-day history of the execution of the script. Without saving this log information, it would be difficult to debug any problems that occur. The next couple of lines simply write out a date/time stamp.

In lines 5-10, the script confirms that the portable USB drive is mounted. If it is not mounted, there is no reason to continue processing. Depending on your particular disk drive, line #6 will probably have to be changed. To determine what string to use for your own disk drive, plug the drive in and start up a terminal window (Applications → System Tools → Terminal). Enter the command “df” and you will see something similar to the output shown in Fig.2.

Note the last line. This is a reference to my portable disk drive. You will probably have something else after the string “/media/”. Line #6 searches the output from the “df” command looking for the string “PASSPORT”. If it doesn't find it, the script terminates. Change this line appropriately.

The next part of the script ascertains that the music source is available. My music server is a Linux box called 'fedora'. If fedora's /music directory is not mounted, the script mounts the music directory by executing script mount-music (line 16). This script contains just one line:

mount -o ro fedora:

/joe0/music /music

If

your music collection resides locally on your Ubuntu box, then you can simply delete lines 11-22.

You'll have to modify line 23, the line that actually executes rdiff-backup in order to specify where your music collection is located. The parameters I have specified mean the following (in the order they appear):

--print-statistics

This means just that, various statistics will be printed.

/music

This is the directory that contains the original music.

/media/WD\ PASSPORT/music

This is the output directory, where the backup music will be located.

There are many other parameters to rdiff-backup. Typing:

man rdiff-backup

will show you what commands are available. Alternatively, the web site has extensive documentation.

There are two issues with the program that I feel need be mentioned. The first is a “feature” that I knew nothing about until I found it documented in the FAQ. When I looked at my backup disk drive to see my music files, I found something a bit unsettling. Here's a very brief picture of the directory structure that I found:

Note that every artist and album (the fifth artist is Alison Krauss and her album is entitled “Now That I've Found You”) has funny characters instead of a proper upper case letter. In particular, the characters “;065” are standing in place of the letter “A”. The FAQ explains this as follows:

When backing up from a case-sensitive filesystem to a case-insensitive filesystem (such as Mac's HFS+ or Windows's FAT32 or NTFS), rdiff-backup escapes uppercase characters in filenames to make sure that no files are accidentally overwritten. When a filesystem is case-preserving but case-insensitive, it means that it remembers that a file is named "Foo" but doesn't distinguish between "Foo", "foo", "foO", "fOo", etc. However, filesystems such as Linux's ext3 do treat these names as separate files.

My Passport portable disk drive is formatted as a FAT32 filesystem (I wanted it readable from a Windows box, too). Note that the problem isn't really serious. All the mp3 players such as Rhythmbox use the mp3 file metadata to determine album name and artist name; they do not use the physical file name for anything.

The second issue is one that has occurred a couple of times in the past year or two. Something gets fouled up, and rdiff-backup is unable to work. It reports an error. Again, the FAQ explained how to get around the error and restart the backup processing. It's nothing serious; the actual data -- the music -- is fine, but it's the rdiff-backup support files, which keep track of what has been backed up when, that are messed up.

I automatically run my music synchronization script every day at the same time. This is done by use of Linux's cron software. My crontab line looks like this:

15 9 * * *

/home/jberry/util/run-rdiff-backup.sh

This means run run-rdiff-backup.sh every day at 9:15 in the morning. On occasion, I check the log (see line #2 of the above script) to confirm everything is running correctly. As an exercise, you might want to enhance the script to send an email if there was a problem with the backup.