Earning Trust for Your Email Server

I host my own email server, this in itself is a very odd thing to do in this day and age.  If you want email to come from your domain, Google offers this for free and provides the same interface as Gmail. If you insist on running your own mail server, then setting it up to use your ISP as a smarthost is the easy way to go (very easy with Ubuntu), of course I didn’t take that path.

As an aside, setting up a mail server that uses fetchmail to gather email from the various accounts you have, and using a smarthost configuration to send email does give you most of the benefits of running your own mail server with very few headaches.  The reason to do this might be that you don’t want to trust Google (or someone else) to hold all your email, and/or you don’t want the individual PCs in your house to be the storage for your email (hard to migrate to a new machine, recover from disaster).  This is how I started down the path of running my own true email server. [I keep thinking that someone should create an easy to install NAS add-on that provides exactly this type of email server]

Ok, maybe you don’t want to run your own email server but you’re interested in knowing what is involved… Having a static IP address is handy, mostly to save you from DNS issues.  While you can manage to have a domain name tied to a dynamic IP, many blacklists include the IP ranges used by ISP for dynamic addresses.  Of course you need a domain name, and a DNS server too.  You might also want to consider a secondary MX record, in case your connection goes down.  You’ll also want to check that your ISP isn’t blocking port 25 outgoing, and having a valid reverse DNS is important too.

So you’ve followed the Ubuntu documentation and setup a mail server, great.  Assuming your IP address is “clean” (ie: not on a blacklist), then you can probably send email just fine.  Until you start hitting problems where spam filters have taken a dislike to your system – in my case it was Rogers (email provided by Yahoo) that treating my outgoing as spam.  One solution is to have the recipient add your email address to their address book so they do still get to see your email.  It may still get tagged as [Bulk] but it won’t get lost.  This isn’t a great solution for someone new you want to contact, or a friend who isn’t terribly technical.

It turns out there are some additional measures you can take on the email server side to add more trust.  There are three I’ve implemented:

All of them rely on the same basic ‘trick’ of adding a TXT record to your DNS information that serves to validate the email.  This works for the simple reason that spammers tend to use botnets made up of machines without valid DNS records.  SPF simply is a declaration that the IP address sending the email is allowed to send email for the specified domain.  DKIM is an updated version of DomainKeys, but both can be used concurrently and some systems only know one.  Both DKIM and DomainKeys have the email server sign the email with a secret (private) key, and the DNS record has a public key that will validate the signature.

After implementing all three, it turns out Yahoo was still tagging my email as spam.  Very frustrating.  One solution I did consider was to avoid the problem entirely and selectively smarthost email going to rogers.com (and yahoo.com, etc).  In the end, it turns out that Yahoo maintains their own blacklist of sorts and you can request to be removed.  To check this, you need access to a yahoo email account that you can send test messages to.  By examining the header you will see X-YahooFilteredBulk if your IP is on their blacklist, this appears to be independent of the status of your SPF/DKIM/DomainKeys authentication that should show as a pass.  The solution is to fill in the Yahoo form, and be persistent.  Much of the form will not apply but you do need to fill it in with something reasonable (and valid).  After a couple of exchanges over several days I was rewarded with this reply:

While we cannot fully exempt your mail server from our SpamGuard
technology, we have however, made appropriate changes to this IP address
in our database. This should help with delivering mail to the
appropriate Yahoo! folders.

Now email sent to yahoo.com is not tagged as spam or [Bulk] – I did a little victory dance once this happened.

The remainder of this post goes into some of the details of getting the three (SPF, DKIM, DomainKeys) implemented.

Continue reading “Earning Trust for Your Email Server”

Wake On Lan

The ability to wake a machine up over the network is really nice.  This feature is referred to as wake on lan (WOL).  Most modern hardware supports this, and many operating systems have it nicely integrated (Mac OSX for one) – under Linux it required a bit more magic.  This post is specific to Ubuntu 9.10, however it probably applies to other versions and variants.

Primarily I was looking to wake up from suspended state as I tend to let my machine sleep when I’m not using it, however WOL works with both hibernate and full shutdown. If you are having trouble, its a good idea to validate that your BIOS settings are correct – as support for WOL can be enabled/disabled in the BIOS.

Unfortunately since everyone has different hardware, the details of the solution will be different.  I learned a lot of this from a thread in the ubuntu forums.  Here is my specific solution, read on past the break if you want to understand how I arrived at this solution.

I created a file /etc/init.d/wakeonlan with the following contents

#! /bin/sh
### BEGIN INIT INFO
# Provides:          wake on lan
# Required-Start:    $network
# Required-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Configures WOL
# Description:       Configures Wake-On-Lan
### END INIT INFO
#
ethtool -s eth0 wol g
echo enabled >  /sys/class/net/eth0/device/power/wakeup
echo SLOT > /proc/acpi/wakeup
echo KBC > /proc/acpi/wakeup

Ensuring the permissions allow execute (chmod a+x wakeonlan).  And added it to the startup sequence:

sudo update-rc.d -f wakeonlan defaults

Now every reboot you’ll get WOL setup, and the keyboard will wake up the machine too.

Continue reading “Wake On Lan”

Ubuntu 9.10

I run Ubuntu on my personal desktop at home, I started back with 6.10 (Edgy) and have simply used the upgrade distro option to drag things forward into the future.  For various reasons my partition table was sort of messed up, and I had left 42Gb of the drive to WindowsXP – so I’ve been meaning to wipe and re-install for a while.

The 1st step was to back up my existing configuration (backups are good).  Then I punted a bunch of stuff from the WindowsXP partition to let me shrink it with GParted.  I created a bootable USB key using UNetbootin (run under my previous Linux install) – very slick.

ntfs-resizeThe NTFS resize option of GParted is scary, but oh so very useful.

The installer has improved, the partitioning step is still something that may cause you to stop and scratch your head in confusion (not all bad I suppose).  The disk selection suggestion was a bit off for my setup, but I’m not exactly typical.  Installing from a USB key is very nice, I won’t be burning any more install CDs unless I’m forced to.

Post installation fun: Even with a backup, you know wiping a disk and starting over was going to be lots of forehead slapping doh moments.  The built in email client (evolution) has a backup/restore facilty – shame I didn’t think about using it prior to the move. This isn’t a big deal as all of my email lives on my IMAP server, so my only ‘loss’ is some address book entries.

It seems that 9.10 uses a newer version of grub.  I had the USB drive with my backup mounted, and it seems that the update-grub command managed to find my old kernels too? (duh, because the USB drive appears as /dev/sdc1 it is a valid boot drive to look on) Neat, but not what I wanted.  Renaming the /boot directory on the backup drive and running update-grub fixed me.

The Linux version of Chromium has been getting slowly better.  I find it is faster to start than Firefox, but less functional (for now).

Another key add-on for me is VMWare Player.  They have a pretty nice 80MB shell script “installer” (.bundle) that just works under Ubuntu 9.10 – remember to run it as root.

There are lots of other little things still missing: MythTV, Calibre, K3B, Grip, LightScribe software, and many desktop customizations.  However, for the most part with once email was configured and I had VMWare setup to run my virtualized images it was starting to feel like home.

Ideally it would be nice to have a post-new-distro script which would allow me to run and re-create my preferred environment.  I wonder how hard it would be to create one of those..