Category: How To

Mirrored drives are also known as a RAID 1 configuration.  It is important to note that running mirrored drives should not be used as a substitute for doing backups.  My motivation for running a RAID 1 is simply that with the drive densities today, I expect these drives to fail.  A terabyte unit is cheap enough that multiplying the cost by two isn’t a big deal, and it gives my data a better chance of surviving a hardware failure.

I purchased two identical drives several months apart – in the hopes of getting units from different batches. I even put them into use staggered by a few months as well.  The intent here was to try to avoid simultaneous failure of the drives due to similarities in manufacture date / usage.  In the end, the environment they are in is probably a bigger factor in leading to failure but what can you do?

Linux has reasonable software raid support.  There is a debate of the merits of software raid vs. hardware raid, as well as which level of raid is most useful.  I leave this as an exercise up to the reader.  The remainder of this posting will be the details of setting up a raid 1 on a live system.  I found two forum postings that talked about this process, the latter being most applicable.

We will start with the assumption that you do have the drive physically installed into your system.  The first step is to partition the disk.  I prefer using cfdisk, but fdisk will work too.  This is always a little scary, but if this is a brand new drive it should not have an existing partition table.  In my scenario I wanted to split the 1TB volume into two partitions, a 300Gb and a 700Gb.

Now let’s use fdisk to dump the results of our partitioning work:

$ sudo fdisk -l /dev/sdd

Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Device Boot         Start         End      Blocks   Id  System
/dev/sdd1               1       36473   292969341   83  Linux
/dev/sdd2           36474      121601   683790660   83  Linux


Next we need to install the RAID tools if you don’t have them already:

$ sudo apt-get install mdadm initramfs-tools

Now recall that we are doing this in a live system, I’ve already got another 1TB volume (/dev/sda) partitioned and full of data I want to keep. So we’re going to create the RAID array in a degraded state, this is the reason for the use of the ‘missing’ option. As I have two partitions I need to run the create command twice, once for each of them.

$ sudo mdadm --create --verbose /dev/md0 --level=mirror --raid-devices=2 missing /dev/sdd1
$ sudo mdadm --create --verbose /dev/md1 --level=mirror --raid-devices=2 missing /dev/sdd2

Now we can take a look at /proc/mdstat to see how things look:

$ cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sdd2[1]
683790592 blocks [2/1] [_U]

md0 : active raid1 sdd1[1]
292969216 blocks [2/1] [_U]

unused devices: <none>

Now we format the new volumes. I’m using ext3 filesystems, feel free to choose your favorite.

$ sudo mkfs -t ext3 /dev/md0
$ sudo mkfs -t ext3 /dev/md1

Mount the newly formatted partitions and copy data to it from the existing drive. I used rsync to perform this as it is an easy way to maintain permissions, and as I’m working on a live system I can re-do the rsync later to grab any updated files before I do the actual switch over.

$ sudo mount /dev/md0 /mntpoint
$ sudo rsync -av /source/path /mntpoint


Once the data is moved, and you need to make the new copy of the data on the new degraded mirror volume the live one. Now unmount the original 1TB drive. Assuming things look ok on your system (no lost data) now we partition that drive we just unmounted (double and triple check the device names!) and format those new partitions.

All that is left to do is add the new volume(s) to the array:

$ sudo mdadm /dev/md0 --add /dev/sda1
$ sudo mdadm /dev/md1 --add /dev/sda2

Again we can check /proc/mdstat to see the status of the array. Or use the watch command on the same file to monitor the progress.

$ cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sdd2[1]
683790592 blocks [2/1] [_U]

md0 : active raid1 sda1[2] sdd1[1]
292969216 blocks [2/1] [_U] [>....................] recovery = 0.6% (1829440/292969216) finish=74.2min speed=65337K/sec

unused devices: <none>

That’s all there is to it.  Things get a bit more complex if you are working on your root volume, but in my case I was simply mirroring one of my data volumes.

I host my own email server, this in itself is a very odd thing to do in this day and age.  If you want email to come from your domain, Google offers this for free and provides the same interface as Gmail. If you insist on running your own mail server, then setting it up to use your ISP as a smarthost is the easy way to go (very easy with Ubuntu), of course I didn’t take that path.

As an aside, setting up a mail server that uses fetchmail to gather email from the various accounts you have, and using a smarthost configuration to send email does give you most of the benefits of running your own mail server with very few headaches.  The reason to do this might be that you don’t want to trust Google (or someone else) to hold all your email, and/or you don’t want the individual PCs in your house to be the storage for your email (hard to migrate to a new machine, recover from disaster).  This is how I started down the path of running my own true email server. [I keep thinking that someone should create an easy to install NAS add-on that provides exactly this type of email server]

Ok, maybe you don’t want to run your own email server but you’re interested in knowing what is involved… Having a static IP address is handy, mostly to save you from DNS issues.  While you can manage to have a domain name tied to a dynamic IP, many blacklists include the IP ranges used by ISP for dynamic addresses.  Of course you need a domain name, and a DNS server too.  You might also want to consider a secondary MX record, in case your connection goes down.  You’ll also want to check that your ISP isn’t blocking port 25 outgoing, and having a valid reverse DNS is important too.

So you’ve followed the Ubuntu documentation and setup a mail server, great.  Assuming your IP address is “clean” (ie: not on a blacklist), then you can probably send email just fine.  Until you start hitting problems where spam filters have taken a dislike to your system – in my case it was Rogers (email provided by Yahoo) that treating my outgoing as spam.  One solution is to have the recipient add your email address to their address book so they do still get to see your email.  It may still get tagged as [Bulk] but it won’t get lost.  This isn’t a great solution for someone new you want to contact, or a friend who isn’t terribly technical.

It turns out there are some additional measures you can take on the email server side to add more trust.  There are three I’ve implemented:

• Sender Policy Framework (SPF)
• Domain Keys Identified Mail (DKIM)
• Domain Keys

All of them rely on the same basic ‘trick’ of adding a TXT record to your DNS information that serves to validate the email.  This works for the simple reason that spammers tend to use botnets made up of machines without valid DNS records.  SPF simply is a declaration that the IP address sending the email is allowed to send email for the specified domain.  DKIM is an updated version of DomainKeys, but both can be used concurrently and some systems only know one.  Both DKIM and DomainKeys have the email server sign the email with a secret (private) key, and the DNS record has a public key that will validate the signature.

After implementing all three, it turns out Yahoo was still tagging my email as spam.  Very frustrating.  One solution I did consider was to avoid the problem entirely and selectively smarthost email going to rogers.com (and yahoo.com, etc).  In the end, it turns out that Yahoo maintains their own blacklist of sorts and you can request to be removed.  To check this, you need access to a yahoo email account that you can send test messages to.  By examining the header you will see X-YahooFilteredBulk if your IP is on their blacklist, this appears to be independent of the status of your SPF/DKIM/DomainKeys authentication that should show as a pass.  The solution is to fill in the Yahoo form, and be persistent.  Much of the form will not apply but you do need to fill it in with something reasonable (and valid).  After a couple of exchanges over several days I was rewarded with this reply:

While we cannot fully exempt your mail server from our SpamGuard
technology, we have however, made appropriate changes to this IP address
in our database. This should help with delivering mail to the
appropriate Yahoo! folders.

Now email sent to yahoo.com is not tagged as spam or [Bulk] – I did a little victory dance once this happened.

The remainder of this post goes into some of the details of getting the three (SPF, DKIM, DomainKeys) implemented.

Continue reading “Earning Trust for Your Email Server”