One of the challenges I faced in configuring my new backup system on my Ubuntu Linux-based server was setting up my two external drives to mount at the same place in the filesystem.

Why? Because the program I use to manage my backups, BackupPC, expects the storage “pool” to always be in the same place. My options were to write a script to change the BackupPC configuration each time a different drive was plugged in, or make the pool mount in the same location regardless of which drive I’m using. Because I was working with LUKS-encrypted volumes I faced additional complexities which made it  difficult to use other methods (like volume labeling) because of how Gnome handles and mounts such encrypted volumes.

While it took a while to make it work, once I figured it out it did exactly what I needed. Now, I’m sharing my results with you to save you the trouble of doing all the research yourself!

I found the easiest solution both for this problem and my next one (making eSATA drives auto mount) was to use HAL configuration files. HAL stands for “Hardware Abstraction Layer” and is one of the ways Linux interfaces with various hardware elements, including hard drives. Using XML it is easy to create custom policies to define how HAL handles particular devices.

(*Note: I have read that Ubuntu will soon be moving away from using HAL, possibly as soon as 9.10, Karmic Koala. I’ve tested this on Ubuntu 8.04, 8.10, and 9.04, but it should work with any distro that (still) uses HAL.)

(** Note 2: Having recently begun the process of rebuilding my server on Ubuntu 9.10/Karmic, I can verify that this method DOES NOT WORK. I am currently in the process of figuring out a solution, likely using udev and scripting, and will post my results as a post here …)

These configuration files are located in /etc/hal/fdi/policy. My installation included one existing file – preferences.fdi – which defines whether non-removable drives are auto-mounted. (We’ll get to eSATA auto mounting soon enough …)

Some quick notes on how .fdi files work: first devices are matched with pre-determined keys, defined through XML. Then, actions are taken by changing (or “merging”) other keys. A very basic .fdi file might consist of the following:

<?xml version="1.0" encoding="ISO-8859-1"?> <!-- -*- SGML -*- -->
<deviceinfo version="0.2">

 <device>
   <match key="some.key" string="some_string">
     <merge key="another.key" type="bool">true</merge>
   </match>
 </device>

So what’s the easiest way to find these keys to match and merge? In my experience, Gnome Device Manager (aka gnome-device-manager).

To see if you have Device Manager installed, check under “Applications -> System Tools” or try to launch gnome-device-manager from the terminal. If it doesn’t work, install it using Synaptic or run:

sudo apt-get install gnome-device-manager

With your drive plugged in and turned on, open up the device manager and begin to look for your drive. You may have to expand some of the trees for “SCSI Host Adapters” and “SCSI Devices”, or possibly USB-related trees,  before you see the drive entries, usually titled “Mass Storage Drive” or something about removable storage.  There may be several of these entries. You will need to look at attributes like the model numbers and capacities to determine which drive it is you’re wanting.

Once you’ve identified the drive you want the information for, type ctrl+p or click on “View -> Device Properties” which should make an additional “Properties” tab appear beside the previous “Summary” tab. Now you will be able to see the various keys, types, and values you can use when creating .fdi files. For our purposes here we will need to look at the properties not for the drive itself, but for the volume we want to create a mount point for.

Note: this is with drive attached via eSATA

Our first order of business is to find a key that will provide a definitive match for each drive / volume. In my situation I was using two identical external enclosures with two identical drives. I could have simply created one rule that would specify the mount point based on any drive that matched some shared property, such as the model number (storage.model) but I wanted to make sure that these two volumes and only these volumes would share this mount point. In addition, I wanted to be able to tell at a glance which one was mounted, so I needed HAL to distinguish between them somehow.

Every volume has a distinctive UUID – kind of like a fingerprint that identifies it to the rest of the system. In the device manager, under the properties tab for the volume you’re working on, look for the volume.UUID key. Once you’ve found it you can begin to write your HAL policy file.

Create a new, blank text file in /etc/hal/fdi/policy. (You will probably need to do all this as root / sudo) Name it something like 30-sharedmounts.fdi. (The “30″ in the file name makes sure this policy is evaluated prior to the general preferences file.) Once you have created your file, open it in your favorite text editor and start with the following:

<?xml version="1.0" encoding="ISO-8859-1"?> <!-- -*- SGML -*- -->
<deviceinfo version="0.2">

Now we need to specify the device and they keys we want to match and merge:

<device>
 <match key="block.is_volume" bool="true">
 <match key="volume.uuid" string="7feeefbf-416b-4383-9bb3-7fd51cb3e702">
   <merge key="volume.policy.desired_mount_point" type="string">ext_backup</merge>
   <merge key="volume.label" type="string">Ext Backup 1</merge>
 </match>
 </match>
</device>

Here’s a quick explanation of what you see above, and what each does:

• <match key=”block.is_volume” bool=”true”> – A little bit of insurance, to make sure we’re working with a volume and not a drive.
• <match key=”volume.uuid” string=”7feeefbf-416b-4383-9bb3-7fd51cb3e702″> – As described above, identifies the particular volume you want to specify the mount point for.
• <merge key=”volume.policy.desired_mount_point” type=”string”>ext_backup – This is the new mount point, which will appear beneath /media/ , so this example would mount as /media/ext_backup .
• <merge key=”volume.label” type=”string”>Ext Backup 1 – A little extra nicety. This is how we can identify between different volumes, even when they’re mounted at the same point in the filesystem.

Now that you’ve got the mount point specified for your first volume, you’ll want to do the same for your second (and any other subsequent) volumes. For each volume, just find the UUID and create a new <device> section in the .fdi file. Be sure to specify the same mount point and unique volume labels (if you so desire.) Once you’re done, save your file and restart HAL (or just reboot.)

sudo /etc/init.d/hal restart

Now, whenever you plug in your drives and Gnome auto-mounts your volumes, they will mount in the same place every time.

What’s that, you say? You’re using eSATA drives and they don’t auto-mount? My next blog post will take care of that for you, and it shouldn’t take more than about 5 minutes of your time. As an added bonus, this method will mount the volumes in the same place regardless of whether they’re connected by USB or eSATA.

After obtaining (and assembling) my new backup hardware my first order of business was to get the external hard drives formatted and set up with drive encryption. Since I’m swapping out one drive to store off-site I wanted to use drive encryption just in case it somehow ended up in someone else’s posession. In Ubuntu, the easiest way to encrypt a whole drive is using LUKS. I chose LUKS because it is easily readable by most Linux computers and can even be set up on a Windows PC, if absolutely necessary. This post will walk through the (remarkably simple) process of setting up encryption and formatting.

(Note: most of these instructions are condensed adaptations of articles I found here and here. )

FYI: I set up my drives using these instructions on Ubuntu 8.10 – Intrepid Ibex. I was later able to successfully mount and access the drives using Ubuntu 8.04 – Hardy Heron on my server and 9.04 – Jaunty Jackalope on my laptop.

When setting up my drives I was using USB 2.0, therefore these directions reflect that. The process may have been faster using eSATA, but at the time I did not yet have a controller card. The directions should be applicable to eSATA as well (unless otherwise noted), but your mileage may vary. These instructions can also be easily adapted for creating a handy, encrypted USB “thumb” drive.

Necessary Software

In order to proceed you must have the cryptsetup package installed:

sudo apt-get install cryptsetup

Finding the drive

After powering on the drive and hooking it up to the computer you need to identify the device:

dmesg | tail -20

[33884.688746] usb 4-1: new high speed USB device using ehci_hcd ...
[33884.764079] usb 4-1: configuration #1 chosen from 1 choice
[33884.764868] scsi8 : SCSI emulation for USB Mass Storage devices
[33884.765316] usb-storage: device found at 9
[33884.765321] usb-storage: waiting for device to settle before scan...
[33888.042416] usb-storage: device scan complete
[33888.043707] scsi 8:0:0:0: Direct-Access     HDS72505 0KLA360 ...
[33888.047550] sd 8:0:0:0: [sdb] 976773168 512-byte hardware sectors
[33888.048292] sd 8:0:0:0: [sdb] Write Protect is off
[33888.048300] sd 8:0:0:0: [sdb] Mode Sense: 00 38 00 00
[33888.048305] sd 8:0:0:0: [sdb] Assuming drive cache: write through
[33888.049648] sd 8:0:0:0: [sdb] 976773168 512-byte hardware sectors
[33888.050421] sd 8:0:0:0: [sdb] Write Protect is off
[33888.050428] sd 8:0:0:0: [sdb] Mode Sense: 00 38 00 00
[33888.050432] sd 8:0:0:0: [sdb] Assuming drive cache: write through
[33888.050438]  sdb: unknown partition table
[33888.066470] sd 8:0:0:0: [sdb] Attached SCSI disk
[33888.066545] sd 8:0:0:0: Attached scsi generic sg2 type 0

In the example above (from this article)  you can see that the drive has been recognized as /dev/sdb. Your drives may show up differently (mine appeared as /dev/sdd and /dev/sde.) I’ll continue to use /dev/sdb to refer to the drive we are working with, but you should replace it with whatever your result is.

Create the partition

Before you can actually set up encryption or format the drive, you must create a partition. This is simply a portion of the drive you intend to store data on. A single physical drive may contain multiple partitions (as is usually the case with a linux boot drive) or just one. At this point you could easily choose to set up both an encrypted and a non-encrypted partition on your drive. I didn’t have a need for this, so I’ll be continuing with a single partition.

While it can be accomplished via the command line, I chose to use the graphical GParted program, available under the “System -> Administration -> Partition Editor” menu.

Choose your device via the drop-down menu in the upper right-hand corner. Then, select the unallocated space and create a new partition that encompasses the entire available space. We do not want to format the partition, only create it, so select “unformatted” as the filesystem.

Click “Apply” and your new partition will be created. You should now have a partition named something like /dev/sdb1 (notice there’s now a number added.) Once you have created the partition successfully, close GParted.

Setting up encryption

The next section is copied verbatim from the aforementioned article:

The dm-crypt, sha256 and aes kernel modules will need to be loaded prior to encrypting the partition:

sudo modprobe dm-crypt
sudo modprobe sha256
sudo modprobe aes

If the following error messages appear when loading sha256 and aes:

sudo modprobe sha256
WARNING: Error inserting padlock_sha ... No such device

sudo modprobe aes
WARNING: Error inserting padlock_aes ... No such device

it is an indication that the system does not have a hardware cryptographic device (source: Ubuntu Bug #206129)

The workaround is to add the following lines (using your favorite editor) to the bottom of /etc/modprobe.d/aliases and re-run the modprobe commands for the sha256 and aes kernel modules:

alias sha256 sha256_generic
alias aes aes_generic

Note that this is only necessary when we are setting up the drive. Later we will access them through Gnome and won’t need these modules.

Encrypting the partition

Finally, we can run the command to encrypt the /dev/sdb1 partition.

While there are other ways of securing your encrypted drive (such as key file stored locally or on a USB flash drive) I have chosen to use a strong passphrase. Mine is 23 characters long, but any password 12 characters or longer not consisting of dictionary-findable words should suffice.

The tutorial I used recommended the following command:

sudo cryptsetup --verify-passphrase luksFormat /dev/sdb1 -c aes -s 256 -h sha256

The LUKS-formatting command above has the following options:

• –verify-passphrase – ensures the passphrase is entered twice to avoid an incorrect passphrase being used
• -c aes – specifies the use of AES encryption (c for cipher)
• -s 256 – specifies a 256-bit key size
• -h sha256 – use 256-bit SHA for password hashing

However, after reading this article I decided to go with 128-bit AES encryption instead, along with a strong passphrase, in hopes of reducing some computational overhead. Therefore, my command instead looked something like:

sudo cryptsetup --verify-passphrase luksFormat /dev/sdb1 -c aes -s 128 -h sha256

Creating the filesystem

After setting up the encrypted partition, you must open and map it in order to set up the filesystem and begin using it.

Start with:

sudo cryptsetup luksOpen /dev/sdb1 secureUSB

Which should prompt you for your passphrase and map the drive to /dev/mapper/secureUSB. Now you can format the encrypted partition with a filesystem using whatever method you prefer. I had good luck using GParted to format in ext3.

Like before, select the device you want to create the filesystem on – in this case /dev/mapper/secureUSB. You should again see a block of unallocated space, which you should select and create a new partition within. You may be presented with a big, scary message about setting a disklabel – tell it to create (since you have no data to lose on the drive at this point anyway!)

The create new partition screen will look just the same as before, only this time you will want to specify the filesystem type you want to use. Apply all the pending operations and wait for the formatting process – it can take quite a while, especially for large drives.

Mounting the encrypted drive

Having successfully set up encryption and created a filesystem, we’re almost ready to to mount the drive and begin using it! (Of course, this is still assuming you’re using USB. If you’re using eSATA then this won’t work as easily – that will come in a later post!)

Shut down the computer, disconnect the drive, and reboot. Once you’re back up and logged in, reconnect the drive and Gnome should prompt you for the passphrase and then mount automatically.

If you want the drive to unlock automatically on this computer, select “remember forever.”

After you’re unlocked and mounted, the final step is to take ownership of the drive’s root folder with a user other than your sudo/root user:

sudo chown youruser:youruser /media/disk

where youruser is the user you want to have ownership and /media/disk is where gnome auto-mounted the drive.

Finished!

Whew! I must say it has taken far longer to write this post than it did to actually perform these operations. The longest part of the entire process was creating the ext3 filesystem. If you’re using eSATA to set up the drive it will go much faster, however as I mentioned you will have a few more steps to take before you can easily hot-swap your drive. Stay tuned for the next post in this series for that how-to …

Over the next couple days (weeks?) you can expect more on the following topics:

• New backup hardware: drives, enclosures, and controller card
• Setting things up: LUKS encryption on external drives
• Specifying a shared mount point in Ubuntu Linux
• Switching from USB to eSATA, making eSATA automount
• Making external encrypted drives work with BackupPC

As you can tell from this list I faced a lot of hoops to jump through in order to get this up and running. I’ll add links to the individual posts as I write them. Feel free to ask questions if you’ve got any!

1. Not reading through the recipe all the way before starting.
2. Overcrowding the pan.
3. Not preheating the pan.
4. Not using a large enough pot for cooking pasta.
5. Sautéing wet greens.
6. Substituting dried herbs for fresh without properly compensating.
7. Frying food in oil that is not hot enough

Even as an experienced home cook I sometimes fall prey to a few of these easy-to-make mistakes, such as pan overcrowding and not waiting long enough for oil to come up to temperature. In addition to naming each mistake the article also goes on to give tips on how to avoid making the mistake in the future. It’s well worth reading, so check it out:

7 Common Cooking Mistakes – Real Simple

All over the internet, little orange icons (like the one on the right) and invitations to “subscribe” to web sites are popping up everywhere. You may wonder what this is all about. RSS stands for “Really Simple Syndication.” But what is most important is that it literally turns the experience of surfing the web upside down.

Think of how you go about browsing the web now. You might go to a bunch of different places such as news sites, blogs, forums, or maybe even more “social” sites like MySpace, Flickr, or PicasaWeb. Each time you visit these sites you check to see if there is anything new since the last time you visited. Sometimes there is, sometimes there isn’t. Wouldn’t it be nice to know before you go to those sites whether there is anything new waiting for you? That’s where RSS comes in. By using RSS to subscribe to websites you visit frequently you can make the web come to you.

One great analogy about understanding the concept of RSS that I heard recently is this: think about video rentals. Now days there are two main ways to rent videos. One method requires you to get in your car, drive to the local video rental shop, rent a movie, and then drive home to watch it. The other method, used by companies like Netflix, lets you decide what movies you want to watch, then Netflix will send them to you in the mail to watch whenever you want. RSS does for the surfing the web what Netflix does for movie rentals. You decide what websites you want to see and RSS sends them to you whenever they are updated! Cool.

Keep reading for instructions on how to get started.

Getting Started

In order to get started using RSS, let’s quickly talk about some definitions of words that you might encounter:

Reader – A reader is how you actually receive and read information using RSS. There are many different readers, including separate programs that you run on your computer and sites on the web that serve as readers. Readers are also sometimes referred to as “news readers”, “feed readers”, or “aggregators.”

Feed – A feed is a stream of items from a particular website. Some websites offer many different feeds that contain different items. For example, on this website there is a feed for all items, as well as feeds for each of the main categories.

Subscribe – In order to receive feeds in your reader, you must subscribe to them. Each different feed has an address, just like a web page would (http://blah.blah.blah). Most modern web browsers and feed readers will allow you to subscribe to a feed by just clicking on a link. These links will often use some form of the standard RSS icon: .

The first thing you will need to do is get an RSS reader. Some popular options are Bloglines, NewsGator, NetVibes, and Google Reader. Each of these work slightly differently, but to get started you will need to either download and install the program or sign up for the online service. Another option is to use the built in feed-reading capability of Mozilla Firefox 2 (or newer) or Internet Explorer 7 (or newer.) Firefox uses a feature called “Live Bookmarks”, while IE’s reader works more like a “regular” feed reader. For the rest of this example I’ll be using Google Reader, since it is my favorite reader and widely regarded as one of the best.

The first thing you’ll need to do is sign up for a Google account (if you don’t have one already.) It’s quick and relatively painless and also opens the door for you to be able to use other great Google services such as Gmail and Google Calendar. If you already have a Google account then you should be able to log in to the Google Reader site using your existing email address and password.

Once you’re set up with your reader, now it’s time to set up some subscriptions. Start off by surfing to some of your usual web sites and looking around for RSS icons or feed links. Often these links will be on the side or near the bottom of a web page. A neat feature found in most modern browsers is called “feed autodiscovery.” This allows webmasters to make it even easier for you to subscribe by providing a link to their feeds in the tool bar of your browser (see photo on right.) When you click on any of these feed links your browser will display a preview of the feed. If you are using Mozilla Firefox (and if you’re not … why aren’t you?) the preview page will look like this:

Choose “Google” from the drop-down menu and click “subscribe now.” You will then be asked whether you want to add the feed to your Google homepage or Google Reader. Click “Add to Google Reader.” You should then be taken to the main Google Reader window where it will show that you are now subscribed to a new feed:

There is also a second way to subscribe to new feeds. This way is nearly universal, and the basic process should work regardless of which browser or reader you are using. Follow the same instructions for finding feeds as above, but when you click on the feed link, copy the link text from the address bar. Then open Google Reader (or your reader of choice) and click on “add subscription” or whatever similar option your reader provides. Paste the feed’s address into the box and click “add.” You should now be subscribed to the new feed.

What Now?

Now that you have the basic process down you can begin to add more feeds as you discover them during your regular web surfing. I’ve found that once I start to visit a page more than two or three times a week it is much more efficient to add an RSS feed from that site to my reader. By doing this I’ve discovered that I spend much less time sufing aimlessly and more time with content that I really care about. So get out there and start subscribing! Why not start now with a subscription to this blog?

Happy feed-reading!