Month: May 2012

I’ve flipped back and forth between reading physical books and eBooks over the last couple of years. I’m currently in an eBook phase, and it may stick this time. A sale on Kobo let me grab a few I had been meaning to read for next to nothing, now that I’ve bought a few I’m more likely to buy more.

Sometime you want to move some content into a format that can be easily read using one of the eReaders. Let’s consider two scenarios: a) You have a paper copy of something you want to scan and convert, b) there is a web resource that is formatted as pages but isn’t in PDF format. Under Ubuntu I like Simple Scan, it allows you to easily scan multi-page documents. If dealing with a web resource, a full screen browser window and Alt-Print Screen will perform a screen capture allowing you to save a series of pages quickly.

Simple Scan will save multiple scanned pages with filenames (Scanned Document-1.jpg) which sort nicely in order of scan. The screen shot utility uses filenames in the format  “Screenshot at YYYY-MM-DD HH:MM:SS.png” so again we have perfect alphabetic sorting in the directory. Having the files in the directory in the correct order will be helpful later on.

Now with both scanning and screen capture there will be elements in the image that we want to crop. As we’re likely dealing with 10’s of pages, we don’t want to have to open GIMP on each of them and edit. Enter ImageMagick – a command line friendly tool for image processing. My screen resolution is 1680×1050 and the screen shots were all 1680×1026 (due to the Ubuntu desktop title bar). The screen shot contained the browser “chrome” as well as portions of the page I didn’t want. Using GIMP I was able to determine the upper left (491×126) and lower right (1170×1026) corners of the image, a little math told me the cropped image size was 679×900. I made a copy of one of the images and called it x.png, this let me experiment to make sure I got it right.

$ convert x.png -crop 679x900+491+126 y.png

Excellent, the resulting y.png file is properly cropped. Now I want to convert all of the files in the directory, and in fact I want to mutate them in place. It turns out mogrify is the the solution:

$ mogrify -crop 679x900+491+126 *.png

This will modify all of the images “in place” in the directory I’m using. For scanned images we have pretty much the same process yet the cropping dimensions will be different.

At this point I jumped the gun and converted all of the files in the directory into a pdf. Here is a screen capture of the PDF viewer showing a simple example to demonstrate the problem:

So while the cropped .png displays properly with no whitespace around it, the PDF clearly has additional whitespace. The ImageMagick identify utility helps explain what’s wrong here:

$ identify Screenshot\ at\ 2012-05-29\ 20\:26\:25.png
Screenshot at 2012-05-29 20:26:25.png PNG 679x900 1680x1026+491+126 8-bit DirectClass 1.263MB 0.050u 0:00.050

Ah, so the image still has the original size, but it’s been cropped to the corrected size. It turns out I want to apply an additional processing step to the images, +repage (to completely remove/reset the virtual canvas meta-data from the images)

$ mogrify +repage *.png

$ identify Screenshot\ at\ 2012-05-29\ 20\:26\:25.png
Screenshot at 2012-05-29 20:26:25.png PNG 679x900 679x900+0+0 8-bit DirectClass 1.263MB 0.050u 0:00.050

Now I’m ready to create a PDF file:

$ convert *.png book.pdf

This works like a charm because my files are in the correct order. The resulting PDF size is a little bit bigger than the sum of the individual image files. I did explore ways to reduce this, but all of them resulted in lower quality images in the PDF and that impacted readability.

One of my work machines runs Ubuntu, to protect the data stored on this machine an encrypted file system is used. The file system encryption is LUKS based and is applied to a filesystem at creation time, thus to encrypt the system drive it is applied at install time. In my case encryption was an option built into the installer I used.

After running the system for a while I wanted to add a 2nd drive for additional storage and backup. One solution would be to follow this post or this one, both use a key file stored on the first drive to open the second. Ideally I want the crypt password supplied once on boot to unlock both drives.

It seems from searching around that this can be done, but it isn’t clear if hacking the stock scripts is needed or not. I also found the posts to be somewhat lacking step-by-step details. So I’ll try to provide a better how-to here.

Phase 1 – gather some data about our current system.

Determine which disk I want to change. Be very careful, modifying the wrong physical disk could be very bad.

$ sudo fdisk -l

Disk /dev/sda: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xe3e5464a

Device Boot Start End Blocks Id System
/dev/sda1 * 1 34 273073+ 83 Linux
/dev/sda2 35 30401 243922927+ 83 Linux

Disk /dev/sdb: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x51e1dd3f

Device Boot Start End Blocks Id System
/dev/sdb1 1 30401 244196001 7 HPFS/NTFS

So the plan is to change /dev/sdb into a new encrypted volume.

Let’s now look at the existing encrypted file system to determine how it is configured. I happen to know mine is called lvm_crypt, you should be able to sort this out by looking in /dev/mapper.

$ sudo cryptsetup status lvm_crypt
/dev/mapper/lvm_crypt is active:
cipher: aes-xts-plain
keysize: 512 bits
device: /dev/sda2
offset: 4040 sectors
size: 487841815 sectors
mode: read/write

We’ll want to mimic the cipher and keysize to keep things at the same security level.

Phase 2 – creating an encrypted filesystem

It appears from my experience that the type of the partition doesn’t matter, but for completeness we’ll repartition the drive to be a Linux partition. Again, when doing this be very careful you are specifying the correct disk – it will destroy information.

$ sudo fdisk /dev/sdb

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').

Command (m for help): d
Selected partition 1

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-30401, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-30401, default 30401):
Using default value 30401

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Now we have a newly partitioned disk. The above may seem somewhat cryptic, we’re simply removing the one existing partition and creating a new one. The default fdisk partition type is a Linux partition suitable for our needs.

Now we create the encrypted filesystem on the new partition, supplying additional parameters to match the key size and cipher of the system volume.

$ sudo cryptsetup luksFormat /dev/sdb1 --key-size=512 --cipher=aes-xts-plain

WARNING!
========
This will overwrite data on /dev/sdb1 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase:
Verify passphrase:

I used the same passphrase as the system volume for convenience (and the hope that I could type it in once on boot).

Now we open it and give it a /dev/mapper name, and then format the volume.

$ sudo cryptsetup luksOpen /dev/sdb1 data_crypt
$ sudo mkfs.ext4 /dev/mapper/data_crypt

At this point we could get paranoid and fill the new volume with random data to prevent any latent zeros on the disk from reducing the set of data an attacker would need to examine. I’m not that paranoid about this system.

Phase 3 – mounting the encrypted filesystem at boot time

We’ll be messing with a couple of files: /etc/crypttab, /etc/fstab/ and the initramfs. My working theory is that /etc/crypttab is used to mount the crypt’d filesystems and /etc/fstab is used to mount the parititions (in that order). The initramfs is stored on the unencrypted boot volume and contains a snapshot of the configuration files we need to bootstrap. I found this post somewhat helpful in figuring this part out.

First we use blkid to determine the UUIDs.

$ sudo blkid /dev/sda1
/dev/sda1: UUID="a7357d62-71ad-47d5-89cb-fd0f42576644" TYPE="ext4"
$ sudo blkid /dev/sda2
/dev/sda2: UUID="e4ff5a5f-39f7-4f3e-a45e-737229d95e10" TYPE="crypto_LUKS"
$ sudo blkid /dev/sdb1
/dev/sdb1: UUID="06114da2-138f-401c-9c84-d4a2e6e83bd1" TYPE="crypto_LUKS"

So we add the following line to /etc/crypttab:

data_crypt UUID=06114da2-138f-401c-9c84-d4a2e6e83bd1 none luks

And we now add one line to /etc/fstab:

/dev/mapper/data_crypt        /data        ext4        defaults        0        2

Before we reboot, we need to update the initramfs so these configuration changes will be seen at boot time.

$ sudo update-initramfs -u

Assuming all went as planned, you’re done. Reboot and test it out.

My journey wasn’t “as planned”, I made several silly mistakes. Providing the wrong UUID causing a failure to open the encrypted volume, and using the wrong name in fstab. Both easy to diagnose by reading the error messages (and logs) carefully and walking through the steps manually. Measure twice, cut once has an application here.

I wasn’t successful in getting to a single password entry on boot. I’m prompted twice for the passphrase at boot time, this is easy enough to do. I’m sure I could crawl in and modify the boot scripts to remember and re-try but that’d result in a non-standard configuration causing upgrade pain the in future. Not worth it for a system I rarely reboot.

Phase 4 – bonus marks, testing recovery via LiveCD

I wanted to verify that I could recover from a failure resulting in an inability to boot the system from the hard disk.

This turned out to be really easy. Boot the Live CD, then unlock the volumes:

$ sudo cryptsetup luksOpen /dev/sda2 lvm_crypt
$ sudo cryptsetup luksOpen /dev/sdb1 data_crypt

Now mount them. My system volume is a LVM, and the new volume is just plain old ext4. I found a helpful post on mounting LVMs from rescue mode:

$ sudo lvm vgscan -v
$ sudo lvm vgchange -a y
$ sudo lvm lvs --all
$ sudo mount /dev/mapper/ubuntu-root /mnt

Mounting the ext4 partition is simply

$ sudo mount /dev/mapper/data_crypt /mnt2

That’s it, a second volume added to an existing LUKS system – and confidence we can mount both volumes from a LiveCD in the case of failure.