A couple of years ago I picked up a conference give-away flash drive (4GB), which at the time seemed like a pretty nice freebie. The trouble was it only every liked to play nice with my Windows machine, Linux would refuse to mount it. The headline photo is the final product, I failed to take a before picture but the leather + snap case on this USB thumb drive was hideous anyway.

Turns out it was a fake, good thing it was free. Still in my typical fashion I didn’t want to just throw it away, heck I’m still carrying around the mysterious AMD 1GB key I got ages ago. So this bogus 4GB key sat in my work bag for a long while before I finally got to investigating it.

The very first thing I did was use the linux command lsusb, this helped me clue in that there was something wrong (fake) with the drive. I found a forum post that helped me get started down the right path. I got a copy of ChipGenius which told me the following:


Description: [I:]USB Mass Storage Device(Generic Flash Disk)

Device Type: Mass Storage Device

Protocal Version: USB 2.00
Current Speed: High Speed
Max Current: 100mA

USB Device ID: VID = 0011 PID = 7788
Serial Number: 874BE199

Device Vendor: Generic
Device Name: Mass Storage
Device Revision: 0103

Manufacturer: Generic
Product Model: Flash Disk
Product Revision: 8.00

Controller Vendor: Alcor Micro
Controller Part-Number: SC708(FC8708)/AU6987 - F/W EC23
Flash ID code: ADD5949A - Hynix H27UAG8T2BTR - 1CE/Single Channel [MLC-8K] -> Total Capacity = 2GB


So this felt like progress: it's 2GB and not 4GB as Windows seems to think. Still not bad for free. I then used my camera to get some close up shots of the naked circuit board to confirm the data that the ChipGenius tool dug out.

It was good to see that the values matched, this helped boost my confidence in the ChipGenius tool. I did find some references on the web claiming that sometimes ChipGenius is wrong, so it’s worth looking at the chips themselves.

Part of the output was also a link to a website, yet even with google translate the site left me guessing as to what I wanted to download – there were a lot of possible options. I choose one near the top “Series master, the Alcor MPtool AU6987T/6989 Yasukuni, production tools (2011.12.26.00)” as the title matches some of the data in the ChipGenius dump. In the end the stability of the site, language barrier and my inability to successfully download anything sent me off down other paths.

I then ended up searching on flashboot.ru with the controller chip number (FC8708) I was able to find and download (with a bit of google translate help) a tool that recognized the drive. The best way to find this tool is searching it’s name: FC_MpTool_FC8308_FC8508_FC8406_04.02.01.

The user interface was mysterious, but clicking on the drive letter started a reformat.. which resulted in a 2GB flash drive. This newly formatted drive was quite happy under Linux.

In terms of performance, I benchmarked copying 7 x ~300MB video files (total 1.9GB) to the stick, this reported ~4.7MB/sec. There were certainly bursty updates in file file progress dialog in Ubuntu. This isn’t great, but again it was free and it works under Linux. I also tried zeroing the entire drive ($ sudo dd if=/dev/zero of=/dev/sdc) which reported: 2095054848 bytes (2.1 GB) copied, 335.285 s, 6.2 MB/s – again, not great but good enough.

A bit of heat shrink tubing applied to cover up the bare circuit board and I’ve got a hack worthy USB key.

A couple of weeks ago my APC Back-UPS 1200 XS started beeping at me, the battery light was flashing, a clear sign the battery has packed it in. I wasn’t overly surprised as the unit is quite old (I’m guessing 8 years? I can’t recall exactly when I got it) – you should only really count on a battery to last 3 to 5 years so it’s done well for me.

While I don’t remember exactly when I bought the UPS, I do remember price matching FutureShop against BestBuy, the price difference was only $20 but it still made me laugh to do it as they are both owned by the same parent company – that and the price match gave me an additional 10% off of the difference in savings (yup, a whole $2). The manual says I should buy a RBC33 battery pack, these are nearly the same price I paid for the entire UPS ($99 + tax).

I looked locally and on the web for a reasonably good deal for a replacement battery pack. The prices had quite a range and I could have opted to go for a RBC32 which is cheaper but still nearly the cost of original UPS. In the end I opted to go for the DIY route and just buy compatible batteries and do a swap – ebay had the best prices, but amazon.ca had a vendor that was almost the same price and I thought I’d go that route as shipping would be quicker.

The batteries arrived fairly quickly (about a week), faster than I’d expect from any shipment from the US. The two batteries are an identical size match to the pair that form the APC battery pack. The original battery pack has a wiring harness and the two batteries are stacked with one inverted.

The procedure was very simple, it took me under 10 minutes and I was stopping to take pictures as I went along. Start by peeling away the sticker from the side with the cable sticking out – put it aside if it still is sticky enough to re-use. Fold the batteries so they are side by side. Remove the cables from one battery, then remove the sticker on the other side and separate the batteries. Last remove the harness from the remaining battery.

Reassembly is a matter of working in reverse. I suggest taking pictures as you go as it is a great way to reference which wire went where, but my photos are a reasonable guide as well. You could also watch this youtube video which covers the battery swap.

My completed battery pack looks a lot like a stock RBC33. If the original stickers don’t have any stick left, a little duct tape should work well.

The batteries in the original pack were 9Ah and my replacements are only 8Ah, this will affect the runtime of my UPS – for my needs a few minutes of backup are enough to protect the system, the new battery pack should give me nearly 30 minutes. Generally the power is good in my neighbourhood, and if it is an extended black out we’re without power for hours.

I’ve seen a few battery packs from systems at work where the batteries have bulged and are clearly bad, mine actually looked fine so I may try to restore them later. I did also consider upgrading to sealed marine deep cycle batteries, but the cost was significant and I’m not convinced of the safety of the solution.

One footnote, I have the UPS plugged into my server and use apcupsd to monitor the status. When the old battery had failed and the UPS was beeping my logs filled with the following:

2013-04-07 23:20:49 -0400 Battery reattached.
2013-04-07 23:20:11 -0400 Battery disconnected.
2013-04-07 23:20:11 -0400 Battery reattached.
2013-04-07 23:19:07 -0400 Battery disconnected.

Occasionally the beeping would stop (and I assume the logging) but in a short while it would resume beeping (and logging).

Unfortunately being a secondary computer means that it gets left around (on the floor) and stepped on. This was the sad fate of this particular netbook. It turns out that replacement screens are relatively inexpensive making it well worth fixing. First step was to boot the netbook connected to an external monitor to confirm the resolution of the panel and make sure nothing else was wrong. Armed with that, and a compatibility list  I was able to find a good price for one from a Canadian seller on ebay. Shipping was very fast and the screen came well packaged: boxed and inside a padded envelope.

Disassembly is quite easy, this YouTube video covers it quite well. Below you’ll find some photos and my notes on the process.

(...)
Read the rest of Repair Acer AspireOne 532H Netbook Screen (424 words)

Nothing is quite as sad as a broken bit of equipment. Sadder still when it’s a fairly expensive component and repair costs at a shop are hundreds of dollars. This Nikon DX AF-S 18-55 VR lens took a kid + dog caused tumble to the ceramic floor, it was mounted on the camera and the impact busted the plastic mounting tabs (if you look closely at the picture above you can see the busted tabs).

On the bright side, if the tabs hadn’t given way something else might have taken more impact and broken. By carefully holding the lens on the camera it was possible to verify that it appeared to still be working fine.. if only it would stay mounted. Glue might have worked, but I wasn’t brave enough to slap a little glue onto an expensive lens. A few calls to local camera shops left me a bit shocked at the cost of repairs, it was not worth repairing as a new lens was almost the same price. What I found interesting was that even buying the parts from a local shop wasn’t an easy option, it seems Nikon is making it more difficult for repair shops to get parts.

I did find some reasonable web resources out there giving step-by-step information on replacing the mount. You can also pick up the required part via ebay. It turns out you can order the part directly from Nikon as well. Their email support was responsive within 24hrs, and I was able to call them directly with a credit card and have it shipped to me in a couple of days.

So with the correct part in hand, it was time to take things apart. I used both #0 and #00 Phillips screwdriver bits, my fingers and patience. I’ll avoid duplicating the instructions found elsewhere and simply provide a few comments on my experience:

• The screws are very, very small. You’ll want to work in a brightly lit area with care not to lose any.
• When reassembling and I was reaching with my fingers to get the tab with the electrical contacts from where it rested “inside” the lens, I managed to get a fingerprint on the lens. Using more caution you can avoid my mistake.
• I messed up the first reassembly, the bayonet part of the new mounting ring needs to be in the ‘right’ place. Refer to the photo below: At about 10 o’clock there is a small metal pin. The bayonet should be clockwise side of that (say 11 o’clock). Everything fit, but the lens didn’t operate correctly on the camera.
• Those metal rings, you don’t want to move them if possible. The good news is that three of the screws you take out fit down through those rings, making it relatively easy to get lined back up if you move them a bit.

I’d never done this type of repair before and I was done well under an hour total. The second re-assembly was 15mins tops. Total cost: $10 + $15 shipping direct from Nikon Canada. If you opt for the ebay route, you may need to transfer the bayonet metal part from the original as most of them only offer the plastic ring – the official part is complete.

With any hardware upgrade (specifically drives) it’s a good idea to capture what the system thinks things look like before you make any changes. For the most part Ubuntu talks about UUIDs for drives, but a couple of places (at least in my install) use the /dev/sd*# names and can trip you up when you shuffle hardware around. Capturing the drive assignments is simply a matter of:

$ sudo fdisk -l | grep ^/dev

Post hardware installation I was surprised at how much of a shuffle the /dev/sd*#‘s changed around. I was glad I had a before and after capture of the data, it also let me identify the new drive pretty easily.

Early in my notes I have “could it be this simple?” and a link to the kernel.org wiki on RAID. It turns out that yes, it really is that simple — but you do need to follow the steps carefully. I did also find an Ubuntu Forum post that was a good read for background too.

The new drive I had temporarily used on an OSX system to do some recovery work, so fdisk wasn’t very happy about working with the drive that had a GUID partition table (GPT). It turns out parted was happy to work with the volume and let me even change it back into something fdisk could work with.

I puzzled over the fact that this new drive wanted to start at 2048 instead of 63, I was initially under the incorrect assumption this had something to do with the GPT setup that I hadn’t been able to fix. Consider two basically identical volumes (old followed by new)

$ sudo fdisk -l /dev/sdb

Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdb1 63 1953520064 976760001 83 Linux

$ sudo fdisk -l /dev/sdc

Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdc1 2048 1953525134 976761543+ 83 Linux

I’ve highlighted the key differences in bold, you can see the physical sector size is 4096 vs. 512 and that is the reason for the different start position. Ok, diversion over – let’s actually follow the wiki and get this drive added to the RAID array.

Start by looking at what we have:

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10]
md_d3 : active raid5 sdf1[1] sdd1[0] sdb1[2]
1953519872 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

So, my RAID5 array is /dev/md_d3, and I know my new drive is /dev/sdc1 after my parted/fdisk adventure above.

$ sudo mdadm --add /dev/md_d3 /dev/sdc1

Now we look at mdstat again and it shows we have a spare. This is honestly what I should have at least done with the drive immediately after installing it – having a spare lets the RAID array fail over to the spare drive with no administrator intervention.

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10]
md_d3 : active raid5 sdc1[3](S) sdf1[1] sdd1[0] sdb1[2]
1953519872 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

Next we grow the array across the new device

$ sudo mdadm --grow --raid-devices=4 /dev/md_d3

You can peek at /proc/mdstat from time to time (or use the watch command) to monitor progress. This may take a while.

Once this is done, don’t forget to modify /etc/mdadm/mdadm.conf as per the wiki: “To make mdadm find your array edit /etc/mdadm.conf and correct the num-devices information of your Array”

At this point we now have our data spread across more drives, but don’t have a larger volume. We need to resize the volume to take advantage of the new space. It’s recommended you do the resize with the RAID5 volume unmounted (offline). I set about to do this and hit problems unmounting the volume: this turned out to be samba holding on to the volume, turning that service off fixed things.

Then I hit a show stopper, the resize2fs command failed:

$ sudo resize2fs -p /dev/md_d3
resize2fs 1.42 (29-Nov-2011)
resize2fs: Device or resource busy while trying to open /dev/md_d3
Couldn't find valid filesystem superblock.

Huh? This is something I’ll one day sort out I suppose, but it really beats me what is going on here. You can resize RAID5 while it’s online too, it’s slower and a bit scarier, but it works.

$ sudo resize2fs /dev/md_d3
resize2fs 1.42 (29-Nov-2011)
Filesystem at /dev/md_d3 is mounted on /stuff; on-line resizing required
old_desc_blocks = 117, new_desc_blocks = 175
Performing an on-line resize of /dev/md_d3 to 732569952 (4k) blocks.

This was followed by a few moments of terror as I realized that I was doing this over a SSH connection – what if the connection is lost? Next time I’ll use screen, or nohup the process.

It was neat to watch the free space on the drive creep upwards. It was running at about 1Gb every 2 seconds. Once this finishes, you’re done. My RAID volume went from 1.9T to 2.8T with the new drive.

The very first thing you need to do at any sign of hard drive failure is run a complete backup. If you really care about some of the data, then I suggest you start with the bits you really care about (photos) and work your way outwards to a full backup. Don’t be shy about “wasting” backup drive space, this might be the last hour (or minutes) of functioning drive. I make this comment from experience watching a full backup stop due to total drive failure part way through the photo directories after successfully backing up a bunch of system files.

You can (as I have) try gddrescue or similar recover tools once you’ve done what you can with the drive in terms of backup. This might get you a little bit more data, but these tools are in my opinion last ditch efforts to salvage failing media. You can try the freezer trick, or even warming the drive up but don’t count on them working. It’s worth repeating: nothing beats regular backups – unless it’s automatic nightly incremental backups with off-site replication.

Replacing the failed drive is simply a matter of finding the manual, and buying a matching drive. If this were my personal system I’d be tempted to upgrade to a SSD, but there is still a big price difference. I booted from the Live USB Ubuntu for the first boot after install, this let me check the hardware was good to go and to peek at the SMART data. It was interesting to see that the SMART data says power cycles=4, I guess they really do test the drives at the factory.

Installing using the Dell Windows install disks went smoothly, the driver installation steps were annoying and kludgey feeling but not too horrible. Time then crawled to a stand still as I worked my way through the updates from Microsoft, one patch set at a time from 2008 (era of the install media) to present day. Many, many, many reboots later I had a clean install that was fully patched and ready to go back. If you find yourself having to do this more than once in a long while, consider using a more advanced technique.

Well, with the exception of the problem of a missing key. Amazingly the lack of the key didn’t prevent you from using it – the ‘T’ worked just fine, it just felt very wrong. I quick trip off to ebay and I was able to locate a suitable replacement. It turns out I needed a K26 type key, and that the same model had at least four variations. This website had a handy guide for selecting the type of key you needed.

Pictured above is what arrived in the mail. A rubber plunger, a plastic hinge and the key cap. First I needed to pop the hinge off of the key, the blade from my swiss army knife did the job worked well. Then I had to puzzle a bit over which way the hinge was supposed to go on the keyboard.

I hope the picture above helps others understand how it is supposed to fit on the keyboard – it was the image I had hoped to find on the internet when I was trying to figure it out. Some of the  youtube videos show using needle nose pliers to do the installation, I found that my fingernails did the job. One the hinge in on, simply plop the rubber plunger in the middle – large base down as pictured in the arrive in the mail shot. The key cap will just snap on when placed on top and pressed down – work the top first, then the bottom.

Replacing the missing key was very satisfying,  not very expensive and the improvement was both cosmetic and functional.