One of the challenges I faced in configuring my new backup system on my Ubuntu Linux-based server was setting up my two external drives to mount at the same place in the filesystem.

Why? Because the program I use to manage my backups, BackupPC, expects the storage “pool” to always be in the same place. My options were to write a script to change the BackupPC configuration each time a different drive was plugged in, or make the pool mount in the same location regardless of which drive I’m using. Because I was working with LUKS-encrypted volumes I faced additional complexities which made it  difficult to use other methods (like volume labeling) because of how Gnome handles and mounts such encrypted volumes.

While it took a while to make it work, once I figured it out it did exactly what I needed. Now, I’m sharing my results with you to save you the trouble of doing all the research yourself!

I found the easiest solution both for this problem and my next one (making eSATA drives auto mount) was to use HAL configuration files. HAL stands for “Hardware Abstraction Layer” and is one of the ways Linux interfaces with various hardware elements, including hard drives. Using XML it is easy to create custom policies to define how HAL handles particular devices.

(*Note: I have read that Ubuntu will soon be moving away from using HAL, possibly as soon as 9.10, Karmic Koala. I’ve tested this on Ubuntu 8.04, 8.10, and 9.04, but it should work with any distro that (still) uses HAL.)

(** Note 2: Having recently begun the process of rebuilding my server on Ubuntu 9.10/Karmic, I can verify that this method DOES NOT WORK. I am currently in the process of figuring out a solution, likely using udev and scripting, and will post my results as a post here …)

These configuration files are located in /etc/hal/fdi/policy. My installation included one existing file – preferences.fdi – which defines whether non-removable drives are auto-mounted. (We’ll get to eSATA auto mounting soon enough …)

Some quick notes on how .fdi files work: first devices are matched with pre-determined keys, defined through XML. Then, actions are taken by changing (or “merging”) other keys. A very basic .fdi file might consist of the following:

<?xml version="1.0" encoding="ISO-8859-1"?> <!-- -*- SGML -*- -->
<deviceinfo version="0.2">

 <device>
   <match key="some.key" string="some_string">
     <merge key="another.key" type="bool">true</merge>
   </match>
 </device>

So what’s the easiest way to find these keys to match and merge? In my experience, Gnome Device Manager (aka gnome-device-manager).

To see if you have Device Manager installed, check under “Applications -> System Tools” or try to launch gnome-device-manager from the terminal. If it doesn’t work, install it using Synaptic or run:

sudo apt-get install gnome-device-manager

With your drive plugged in and turned on, open up the device manager and begin to look for your drive. You may have to expand some of the trees for “SCSI Host Adapters” and “SCSI Devices”, or possibly USB-related trees,  before you see the drive entries, usually titled “Mass Storage Drive” or something about removable storage.  There may be several of these entries. You will need to look at attributes like the model numbers and capacities to determine which drive it is you’re wanting.

Once you’ve identified the drive you want the information for, type ctrl+p or click on “View -> Device Properties” which should make an additional “Properties” tab appear beside the previous “Summary” tab. Now you will be able to see the various keys, types, and values you can use when creating .fdi files. For our purposes here we will need to look at the properties not for the drive itself, but for the volume we want to create a mount point for.

Note: this is with drive attached via eSATA

Our first order of business is to find a key that will provide a definitive match for each drive / volume. In my situation I was using two identical external enclosures with two identical drives. I could have simply created one rule that would specify the mount point based on any drive that matched some shared property, such as the model number (storage.model) but I wanted to make sure that these two volumes and only these volumes would share this mount point. In addition, I wanted to be able to tell at a glance which one was mounted, so I needed HAL to distinguish between them somehow.

Every volume has a distinctive UUID – kind of like a fingerprint that identifies it to the rest of the system. In the device manager, under the properties tab for the volume you’re working on, look for the volume.UUID key. Once you’ve found it you can begin to write your HAL policy file.

Create a new, blank text file in /etc/hal/fdi/policy. (You will probably need to do all this as root / sudo) Name it something like 30-sharedmounts.fdi. (The “30″ in the file name makes sure this policy is evaluated prior to the general preferences file.) Once you have created your file, open it in your favorite text editor and start with the following:

<?xml version="1.0" encoding="ISO-8859-1"?> <!-- -*- SGML -*- -->
<deviceinfo version="0.2">

Now we need to specify the device and they keys we want to match and merge:

<device>
 <match key="block.is_volume" bool="true">
 <match key="volume.uuid" string="7feeefbf-416b-4383-9bb3-7fd51cb3e702">
   <merge key="volume.policy.desired_mount_point" type="string">ext_backup</merge>
   <merge key="volume.label" type="string">Ext Backup 1</merge>
 </match>
 </match>
</device>

Here’s a quick explanation of what you see above, and what each does:

• <match key=”block.is_volume” bool=”true”> – A little bit of insurance, to make sure we’re working with a volume and not a drive.
• <match key=”volume.uuid” string=”7feeefbf-416b-4383-9bb3-7fd51cb3e702″> – As described above, identifies the particular volume you want to specify the mount point for.
• <merge key=”volume.policy.desired_mount_point” type=”string”>ext_backup – This is the new mount point, which will appear beneath /media/ , so this example would mount as /media/ext_backup .
• <merge key=”volume.label” type=”string”>Ext Backup 1 – A little extra nicety. This is how we can identify between different volumes, even when they’re mounted at the same point in the filesystem.

Now that you’ve got the mount point specified for your first volume, you’ll want to do the same for your second (and any other subsequent) volumes. For each volume, just find the UUID and create a new <device> section in the .fdi file. Be sure to specify the same mount point and unique volume labels (if you so desire.) Once you’re done, save your file and restart HAL (or just reboot.)

sudo /etc/init.d/hal restart

Now, whenever you plug in your drives and Gnome auto-mounts your volumes, they will mount in the same place every time.

What’s that, you say? You’re using eSATA drives and they don’t auto-mount? My next blog post will take care of that for you, and it shouldn’t take more than about 5 minutes of your time. As an added bonus, this method will mount the volumes in the same place regardless of whether they’re connected by USB or eSATA.

After obtaining (and assembling) my new backup hardware my first order of business was to get the external hard drives formatted and set up with drive encryption. Since I’m swapping out one drive to store off-site I wanted to use drive encryption just in case it somehow ended up in someone else’s posession. In Ubuntu, the easiest way to encrypt a whole drive is using LUKS. I chose LUKS because it is easily readable by most Linux computers and can even be set up on a Windows PC, if absolutely necessary. This post will walk through the (remarkably simple) process of setting up encryption and formatting.

(Note: most of these instructions are condensed adaptations of articles I found here and here. )

FYI: I set up my drives using these instructions on Ubuntu 8.10 – Intrepid Ibex. I was later able to successfully mount and access the drives using Ubuntu 8.04 – Hardy Heron on my server and 9.04 – Jaunty Jackalope on my laptop.

When setting up my drives I was using USB 2.0, therefore these directions reflect that. The process may have been faster using eSATA, but at the time I did not yet have a controller card. The directions should be applicable to eSATA as well (unless otherwise noted), but your mileage may vary. These instructions can also be easily adapted for creating a handy, encrypted USB “thumb” drive.

Necessary Software

In order to proceed you must have the cryptsetup package installed:

sudo apt-get install cryptsetup

Finding the drive

After powering on the drive and hooking it up to the computer you need to identify the device:

dmesg | tail -20

[33884.688746] usb 4-1: new high speed USB device using ehci_hcd ...
[33884.764079] usb 4-1: configuration #1 chosen from 1 choice
[33884.764868] scsi8 : SCSI emulation for USB Mass Storage devices
[33884.765316] usb-storage: device found at 9
[33884.765321] usb-storage: waiting for device to settle before scan...
[33888.042416] usb-storage: device scan complete
[33888.043707] scsi 8:0:0:0: Direct-Access     HDS72505 0KLA360 ...
[33888.047550] sd 8:0:0:0: [sdb] 976773168 512-byte hardware sectors
[33888.048292] sd 8:0:0:0: [sdb] Write Protect is off
[33888.048300] sd 8:0:0:0: [sdb] Mode Sense: 00 38 00 00
[33888.048305] sd 8:0:0:0: [sdb] Assuming drive cache: write through
[33888.049648] sd 8:0:0:0: [sdb] 976773168 512-byte hardware sectors
[33888.050421] sd 8:0:0:0: [sdb] Write Protect is off
[33888.050428] sd 8:0:0:0: [sdb] Mode Sense: 00 38 00 00
[33888.050432] sd 8:0:0:0: [sdb] Assuming drive cache: write through
[33888.050438]  sdb: unknown partition table
[33888.066470] sd 8:0:0:0: [sdb] Attached SCSI disk
[33888.066545] sd 8:0:0:0: Attached scsi generic sg2 type 0

In the example above (from this article)  you can see that the drive has been recognized as /dev/sdb. Your drives may show up differently (mine appeared as /dev/sdd and /dev/sde.) I’ll continue to use /dev/sdb to refer to the drive we are working with, but you should replace it with whatever your result is.

Create the partition

Before you can actually set up encryption or format the drive, you must create a partition. This is simply a portion of the drive you intend to store data on. A single physical drive may contain multiple partitions (as is usually the case with a linux boot drive) or just one. At this point you could easily choose to set up both an encrypted and a non-encrypted partition on your drive. I didn’t have a need for this, so I’ll be continuing with a single partition.

While it can be accomplished via the command line, I chose to use the graphical GParted program, available under the “System -> Administration -> Partition Editor” menu.

Choose your device via the drop-down menu in the upper right-hand corner. Then, select the unallocated space and create a new partition that encompasses the entire available space. We do not want to format the partition, only create it, so select “unformatted” as the filesystem.

Click “Apply” and your new partition will be created. You should now have a partition named something like /dev/sdb1 (notice there’s now a number added.) Once you have created the partition successfully, close GParted.

Setting up encryption

The next section is copied verbatim from the aforementioned article:

The dm-crypt, sha256 and aes kernel modules will need to be loaded prior to encrypting the partition:

sudo modprobe dm-crypt
sudo modprobe sha256
sudo modprobe aes

If the following error messages appear when loading sha256 and aes:

sudo modprobe sha256
WARNING: Error inserting padlock_sha ... No such device

sudo modprobe aes
WARNING: Error inserting padlock_aes ... No such device

it is an indication that the system does not have a hardware cryptographic device (source: Ubuntu Bug #206129)

The workaround is to add the following lines (using your favorite editor) to the bottom of /etc/modprobe.d/aliases and re-run the modprobe commands for the sha256 and aes kernel modules:

alias sha256 sha256_generic
alias aes aes_generic

Note that this is only necessary when we are setting up the drive. Later we will access them through Gnome and won’t need these modules.

Encrypting the partition

Finally, we can run the command to encrypt the /dev/sdb1 partition.

While there are other ways of securing your encrypted drive (such as key file stored locally or on a USB flash drive) I have chosen to use a strong passphrase. Mine is 23 characters long, but any password 12 characters or longer not consisting of dictionary-findable words should suffice.

The tutorial I used recommended the following command:

sudo cryptsetup --verify-passphrase luksFormat /dev/sdb1 -c aes -s 256 -h sha256

The LUKS-formatting command above has the following options:

• –verify-passphrase – ensures the passphrase is entered twice to avoid an incorrect passphrase being used
• -c aes – specifies the use of AES encryption (c for cipher)
• -s 256 – specifies a 256-bit key size
• -h sha256 – use 256-bit SHA for password hashing

However, after reading this article I decided to go with 128-bit AES encryption instead, along with a strong passphrase, in hopes of reducing some computational overhead. Therefore, my command instead looked something like:

sudo cryptsetup --verify-passphrase luksFormat /dev/sdb1 -c aes -s 128 -h sha256

Creating the filesystem

After setting up the encrypted partition, you must open and map it in order to set up the filesystem and begin using it.

Start with:

sudo cryptsetup luksOpen /dev/sdb1 secureUSB

Which should prompt you for your passphrase and map the drive to /dev/mapper/secureUSB. Now you can format the encrypted partition with a filesystem using whatever method you prefer. I had good luck using GParted to format in ext3.

Like before, select the device you want to create the filesystem on – in this case /dev/mapper/secureUSB. You should again see a block of unallocated space, which you should select and create a new partition within. You may be presented with a big, scary message about setting a disklabel – tell it to create (since you have no data to lose on the drive at this point anyway!)

The create new partition screen will look just the same as before, only this time you will want to specify the filesystem type you want to use. Apply all the pending operations and wait for the formatting process – it can take quite a while, especially for large drives.

Mounting the encrypted drive

Having successfully set up encryption and created a filesystem, we’re almost ready to to mount the drive and begin using it! (Of course, this is still assuming you’re using USB. If you’re using eSATA then this won’t work as easily – that will come in a later post!)

Shut down the computer, disconnect the drive, and reboot. Once you’re back up and logged in, reconnect the drive and Gnome should prompt you for the passphrase and then mount automatically.

If you want the drive to unlock automatically on this computer, select “remember forever.”

After you’re unlocked and mounted, the final step is to take ownership of the drive’s root folder with a user other than your sudo/root user:

sudo chown youruser:youruser /media/disk

where youruser is the user you want to have ownership and /media/disk is where gnome auto-mounted the drive.

Finished!

Whew! I must say it has taken far longer to write this post than it did to actually perform these operations. The longest part of the entire process was creating the ext3 filesystem. If you’re using eSATA to set up the drive it will go much faster, however as I mentioned you will have a few more steps to take before you can easily hot-swap your drive. Stay tuned for the next post in this series for that how-to …

Over the next couple days (weeks?) you can expect more on the following topics:

• New backup hardware: drives, enclosures, and controller card
• Setting things up: LUKS encryption on external drives
• Specifying a shared mount point in Ubuntu Linux
• Switching from USB to eSATA, making eSATA automount
• Making external encrypted drives work with BackupPC

As you can tell from this list I faced a lot of hoops to jump through in order to get this up and running. I’ll add links to the individual posts as I write them. Feel free to ask questions if you’ve got any!