Month: August 2011

Recently it seems email spam levels have been increasing, reading through the news it seems there is some debate if it is really up or not. Either way, I know that my local mailbox has been getting more spam (specifically more binary attachment spam) lately. As I host my own email server, addressing spam levels is something I can do something about at the server level vs. needing to rely on clever filtering by the client.

Hosting your own email server is a little dumb. I like the challenge, and knowing my data is stored on my computer systems is comforting. Most people should just stick with gmail or similar. If you are stubborn like myself, Ubuntu makes it easy to setup your own mail server. There is of course the other details on getting a ISP that allows you to host a server etc. (exercise left to the reader)

One alternative is to run a local mail server that smarthosts through your ISP (or even Google). It can use fetchmail or similar to suck down email from your various accounts too. This would result in your data on your machines and good spam filtering as you offload that problem to the other mail server (say gmail).

Ok, so you’re dumb like me and while spamassassin is doing an ok job, it’d be nice to stop more spam from hitting your mail server. The solution is greylisting, the Ubuntu community docs make it very simple to setup if you’ve got a postfix based mail system setup already.

The concept behind greylisting is very simple. Spammers are lazy and so is spam software. One of the error codes a mail server can answer back is ‘temporary failure’. Greylisting causes the first attempt to deliver a given email message with a temporary failure code, any properly configured mail server will retry after a short period of time (usually minutes). Spam software can’t be bothered to go back, it’s spraying email across a large number of servers and a large number of addresses – a few failures aren’t important. If you want to know more, I encourage you to read through the whitepaper.

The trade off with greylisting is that normal email can be delayed. Postgrey uses an adaptive whitelist to allow frequent valid email to skip the temporary fail sequence. The place you’ll notice delays is when you reset a password, since the email is likely to come from a mail server you don’t often get email from – so it will be delayed by the temporary fail code.

After a day – spam has dropped to zero, and email is still arriving in my inbox. I did have to “wait” for a password reset email that was delayed by 1041 seconds (a bit more than 17mins), the delay time is due to the sending server retry cadence.

Looking at a year of mail traffic on my server, it doesn’t appear that volumes are up that much.

Looking at the weekly graph shows a spike in rejects (due to greylisting), but if you look closely you can see the drop off on viruses and spam (since greylisting prevents those messages from ever being received and processed).

 

 

 

In my recent post I talked about the problems I had with blu-ray playback and Ubuntu. I’m now happy to report that I can accomplish what I originally set out to do: rip and convert blu-ray discs on my Ubuntu system for playback on my mobile devices.

The solution was provided by an update to makemkv (1.6.13), it now supports bus encryption capable drives (BEC). As I wrote last time, installing makemkv on your Ubuntu system is straight forward if you follow the detailed instructions. If you’re a developer familiar with ‘make‘ and are comfortable doing a bit of unix admin stuff, then it’s easy.

Makemkv is shareware: free to try for 30 days, then if you’ve found it useful you’re asked to pay for it. The purchase price is 50 euros. The key you receive is valid for all platforms and all versions (including future ones). The price may seem steep, but compare it to the cost of a blu-ray playback device or a video game and it starts to be very reasonable. The linux version is in beta, and while it is in beta it is free.

Unlike my previous experience with makemkv being unable to process a blu-ray disc, this version happily detected the movie and gave me simple way to rip the movie (25Gb!). The data is being decrypted on the fly so this cuts the read speed down to 3.6x – resulting in a 36 minute job for the 2 hour movie.

There is an option to stream the movie from the original disc. When I initially tried this with mplayer it worked, but was laggy. Using vlc streaming is smooth, but my version (1.0.6) doesn’t support the trhd audio codec so I get the spanish track. Running mplayer with a larger cache (-cache 8192) results in smoother playback, but the same spanish audio track problem. The ripped version of the movie plays back smoothly under mplayer and vlc with no audio issues.

The main reason I installed a blu-ray player in my PC was to enable me to rip and convert the movies for use on my mobile devices. Handbrake is my tool of choice for this type of conversion.

As you can see, Handbrake is quite happy to convert the ripped video. My system can typically do a DVD in 20 minutes from physical media to mobile format. With blu-ray it seems I’m closer to real time (ie: 2hrs for a 2hr movie) due to the amount of data and the two step process. Still in the end, I’ve got a version of the movie I can take with me when I travel.

For folks interested in reading up on blu-ray and AACS, I came across the specification (pdf).

 

 

 

 

 

 

Recently I picked up a bluray drive for my PC. As I use Ubuntu as my desktop it would be nice to be able to play back (or rip & convert) movies I own. There is documentation on this, but it is somewhat non-trivial and didn’t work for me.

The model drive I have is the LG CH12LS28 Black 12x Blu-ray Read 16x DVD+/-R/RW Write Combo Drive LightScribe SATA, an amazing amount of technology for the price. Plug it in and away I go. Bluray discs show up just fine (but I lack the software on Ubuntu to play them back), and the other attributes of the drive all appear to check out just fine.

I started out looking into DumpHD. There are multiple bits of software to install to get things to the point where they should work, but I kept hitting the error:

The given Host Certficate / Private Key has been revoked by your drive.

After banging my head on this particular problem for a bit, I then decided to try out makemkv. Installing it takes a bit of doing, but no more than 5 minutes later I was ready to try it out. The end of the error log looked very similar:

Can't read AACS VID from disc - most likely current AACS host certificate is revoked by your drive
The volume key is unknown for this disc - video can't be decrypted
Failed to open disc


However, what I didn’t notice right away was what made it actually fail – this was dumped near the top of the error log:

Drive 'HL-DT-ST BDDVDRW CH12LS28 1.00' requires AACS bus encryption, disc decryption may fail.
Error 'Scsi error - ILLEGAL REQUEST:COPY PROTECTION KEY EXCHANGE FAILURE - AUTHENTICATION FAILURE' occurred while issuing SCSI command

You’d think I’d remember this cardinal rule of debugging – always look at the very first error! So it turns out that the main issue I’m having is AACS bus encryption.

I found a good description of the problem which I’ll rephrase here. AACS bus encryption is a new twist in the story. Bluray has DRM to protect the contents of the media, to access the contents you need to unlock the DRM. Previous to AACS bus encryption the player software would have the key, pass it to the drive, and the content would stream decoded over the sata bus to the player. This let all sorts of traffic ‘sniffing’ attacks happen to the data on the bus. Newer drives (mine include) support additional encryption of the data over the sata bus to block these sniffing attacks. Both makemkv and DumpHD (currently) depend on these sniffing attacks. There is a thread on makemkv on this topic.

As with any DRM scheme, it can be broken for the simple reason that decoding the disc on a computer is a legitimate thing to do using licensed software. Thus all of the magic to decode the disc can be stored on your computer, it is just a matter of knowing the secret to the trick. Sadly, today there is no licensed software for Linux.

Off to windows I went (the same PC I run Linux on, also has a licensed Windows XP install). I dutifully installed the software that came with my new bluray drive – a free copy of PowerDVD 9. I was very sad to see this as the result:

I’m running a fairly economical setup on a core i3 using onboard graphics. What’s burning me here is the lack of suitable graphics drivers from Intel to support HDCP (yes, more DRM). I can either buy a graphics card that supports the right kind of drivers, or upgrade to Windows Vista (or beyond). While I could use this as the excuse to buy a graphics card, I’m certain there is a way to do this in software.

The solution ended up being a combination of three elements:

1. AnyDVD HD
2. UDF 2.5 Filesystem viewer
3. DAUM POTPlayer

AnyDVD HD solves both the AACS bus encryption and DRM issue. It is a driver that accesses the bluray drive directly (avoiding the bus encryption) and can unlock the DRM. There is a free trial.

Since Windows XP can’t read a UDF 2.5 filesystem, which is how the data on a bluray is stored, we need a utility that allows this to be done.

The DAUM POTPlayer can manage to play back bluray content, assuming it has been decrypted (thanks to AnyDVD HD) and is visible as files on disk (UDF 2.5 filesytem). The result you can see evidence of in the screen capture at the top of this post. The experience isn’t very user friendly, but it works well enough.