Earning Trust for Your Email Server

I host my own email server, this in itself is a very odd thing to do in this day and age.  If you want email to come from your domain, Google offers this for free and provides the same interface as Gmail. If you insist on running your own mail server, then setting it up to use your ISP as a smarthost is the easy way to go (very easy with Ubuntu), of course I didn’t take that path.

As an aside, setting up a mail server that uses fetchmail to gather email from the various accounts you have, and using a smarthost configuration to send email does give you most of the benefits of running your own mail server with very few headaches.  The reason to do this might be that you don’t want to trust Google (or someone else) to hold all your email, and/or you don’t want the individual PCs in your house to be the storage for your email (hard to migrate to a new machine, recover from disaster).  This is how I started down the path of running my own true email server. [I keep thinking that someone should create an easy to install NAS add-on that provides exactly this type of email server]

Ok, maybe you don’t want to run your own email server but you’re interested in knowing what is involved… Having a static IP address is handy, mostly to save you from DNS issues.  While you can manage to have a domain name tied to a dynamic IP, many blacklists include the IP ranges used by ISP for dynamic addresses.  Of course you need a domain name, and a DNS server too.  You might also want to consider a secondary MX record, in case your connection goes down.  You’ll also want to check that your ISP isn’t blocking port 25 outgoing, and having a valid reverse DNS is important too.

So you’ve followed the Ubuntu documentation and setup a mail server, great.  Assuming your IP address is “clean” (ie: not on a blacklist), then you can probably send email just fine.  Until you start hitting problems where spam filters have taken a dislike to your system – in my case it was Rogers (email provided by Yahoo) that treating my outgoing as spam.  One solution is to have the recipient add your email address to their address book so they do still get to see your email.  It may still get tagged as [Bulk] but it won’t get lost.  This isn’t a great solution for someone new you want to contact, or a friend who isn’t terribly technical.

It turns out there are some additional measures you can take on the email server side to add more trust.  There are three I’ve implemented:

All of them rely on the same basic ‘trick’ of adding a TXT record to your DNS information that serves to validate the email.  This works for the simple reason that spammers tend to use botnets made up of machines without valid DNS records.  SPF simply is a declaration that the IP address sending the email is allowed to send email for the specified domain.  DKIM is an updated version of DomainKeys, but both can be used concurrently and some systems only know one.  Both DKIM and DomainKeys have the email server sign the email with a secret (private) key, and the DNS record has a public key that will validate the signature.

After implementing all three, it turns out Yahoo was still tagging my email as spam.  Very frustrating.  One solution I did consider was to avoid the problem entirely and selectively smarthost email going to rogers.com (and yahoo.com, etc).  In the end, it turns out that Yahoo maintains their own blacklist of sorts and you can request to be removed.  To check this, you need access to a yahoo email account that you can send test messages to.  By examining the header you will see X-YahooFilteredBulk if your IP is on their blacklist, this appears to be independent of the status of your SPF/DKIM/DomainKeys authentication that should show as a pass.  The solution is to fill in the Yahoo form, and be persistent.  Much of the form will not apply but you do need to fill it in with something reasonable (and valid).  After a couple of exchanges over several days I was rewarded with this reply:

While we cannot fully exempt your mail server from our SpamGuard
technology, we have however, made appropriate changes to this IP address
in our database. This should help with delivering mail to the
appropriate Yahoo! folders.

Now email sent to yahoo.com is not tagged as spam or [Bulk] – I did a little victory dance once this happened.

The remainder of this post goes into some of the details of getting the three (SPF, DKIM, DomainKeys) implemented.

Continue reading “Earning Trust for Your Email Server”

TED 5000 – Installation

I’ve had my eye on the TED device for quite some time.  It allows you to monitor your power consumption at the whole house level.  I’ve used in the past a Kill-A-Watt which is handy to monitor energy usage by a single device.

The TED 1000 had a closed software package, but showed promise.  The TED 5000 provides a web based API to both view your data – and provides a manner to export it.  The TED 5000 is one of the few devices supported by Google Powermeter, while I don’t plan to make use of the Google service I do hope they figure out how to collaborate with the community better.

Ontario has a Smart Meter plan on the way, but the roll out is taking a while and I really don’t trust them to get the web tools right in the near term.  The TED 5000 is available now, and the data is easy to get to if I want to re-graph it in some way.

I ordered the basic package (TED 5000-G) – there are two basic parts that come with it.  The MTU (Measuring Transmitting Unit) and the Gateway, each packaged in their own little box.  The first (and scary part) is installing the MTU as it it attached to your electrical panel.  The MTU communicates over the power lines to the Gateway (no need to run wires from one to the other).  The Gateway needs to be plugged into your router/network and will aquire an IP address via DHCP.  You interact with the Gateway with a web browser to view/configure the data.

The rest of this posting is about the install procedure, read on for gory details.

Continue reading “TED 5000 – Installation”

iPod Touch firmware upgrade with VMWare

With the recent OS 3.1.1 release it was really only a matter of time before I got around to upgrading again.  My previous firmware upgrade was not very smooth.  As I’ve mentioned previously, I run iTunes on a VMWare image of Windows XP – hosted on my Ubuntu desktop.

During the firmware upgrade process the iPod changes the type of USB device it is – this causes problems for Linux/VMWare.  I had previously thought there was no way to get around these issues, making VMWare based firmware upgrades impossible.  I’m happy to report that I was wrong.

Motivated by the OS 3.1.1 upgrade being free of charge (to suckers users who had purchased 3.0, and only $5 otherwise), I figured I’d take another run at making this work under Ubuntu / VMWare.  I found a post that discussed upgrading the iPhone under VMWare, and it turns out that this was pretty much the magic sauce to make things work.  The steps are shamelessly reproduced here, I cannot take the credit:

Create a file /etc/modprobe.d/blacklist-usb containing:

blacklist snd_usb_audio
blacklist usbhid

Reload udev to refresh the configuration we just changed:

sudo /etc/init.d/udev reload

Forcefully remove the kernel modules:

sudo /sbin/modprobe -r snd_usb_audio
sudo /sbin/modprobe -r usbhid

Perform the upgrade under VMWare.

Reverse these steps after the upgrade to restore your Ubuntu machine to normal:
sudo rm /etc/modprobe.d/blacklist-usb
sudo /etc/init.d/udev reload

Read on for gory details on what I actually did..

Continue reading “iPod Touch firmware upgrade with VMWare”