{"id":1303,"date":"2012-12-30T23:16:47","date_gmt":"2012-12-31T03:16:47","guid":{"rendered":"https:\/\/lowtek.ca\/roo\/?p=1303"},"modified":"2012-12-30T23:16:47","modified_gmt":"2012-12-31T03:16:47","slug":"add-volum-to-raid5-on-ubuntu","status":"publish","type":"post","link":"https:\/\/lowtek.ca\/roo\/2012\/add-volum-to-raid5-on-ubuntu\/","title":{"rendered":"Add drive to RAID5 on Ubuntu"},"content":{"rendered":"

Some time ago I migrated from a RAID1 setup to RAID5<\/a>, this was on the minimum 3 drives. At some point this summer I spotted a good deal on a matching 1TB drive to what I had in my array and bought it. My purchase sat in my desk drawer for a month (or two) then I finally got around to installing it into the server. At least another couple of months went by until I got to adding it to my array – it turns out to be really simple and I’m kicking myself for dragging my feet.<\/p>\n

With any hardware upgrade (specifically drives) it’s a good idea to capture what the system thinks things look like before you make any changes. For the most part Ubuntu talks about UUIDs<\/a> for drives, but a couple of places (at least in my install) use the \/dev\/sd*#<\/code> names and can trip you up when you shuffle hardware around. Capturing the drive assignments is simply a matter of:<\/p>\n

$ sudo fdisk -l | grep ^\/dev<\/code><\/p>\n

Post hardware installation I was surprised at how much of a shuffle the \/dev\/sd*#<\/code>‘s changed around. I was glad I had a before and after capture of the data, it also let me identify the new drive pretty easily.<\/p>\n

Early in my notes I have “could it be this simple?”<\/em> and a link to the kernel.org wiki on RAID<\/a>. It turns out that yes, it really is that simple — but you do need to follow the steps carefully. I did also find an Ubuntu Forum post<\/a> that was a good read for background too.<\/p>\n

The new drive I had temporarily used on an OSX system to do some recovery work, so fdisk<\/a> wasn’t very happy about working with the drive that had a GUID partition table<\/a> (GPT). It turns out parted<\/a> was happy to work with the volume and let me even change it back into something fdisk could work with.<\/p>\n

I puzzled over the fact that this new drive wanted to start at 2048 instead of 63, I was initially under the incorrect assumption this had something to do with the GPT setup that I hadn’t been able to fix. Consider two basically identical volumes (old followed by new)<\/p>\n

$ sudo fdisk -l \/dev\/sdb<\/code><\/p>\n

Disk \/dev\/sdb: 1000.2 GB, 1000204886016 bytes
\n255 heads, 63 sectors\/track, 121601 cylinders, total 1953525168 sectors
\nUnits = sectors of 1 * 512 = 512 bytes
\nSector size (logical\/physical): 512 bytes \/ 512 bytes<\/strong>
\nI\/O size (minimum\/optimal): 512 bytes \/ 512 bytes<\/strong>
\nDisk identifier: 0x00000000<\/code><\/p>\n

Device Boot Start End Blocks Id System
\n\/dev\/sdb1 63<\/strong> 1953520064 976760001 83 Linux<\/code><\/p>\n

$ sudo fdisk -l \/dev\/sdc<\/code><\/p>\n

Disk \/dev\/sdc: 1000.2 GB, 1000204886016 bytes
\n255 heads, 63 sectors\/track, 121601 cylinders, total 1953525168 sectors
\nUnits = sectors of 1 * 512 = 512 bytes
\nSector size (logical\/physical): 512 bytes \/ 4096 bytes<\/strong>
\nI\/O size (minimum\/optimal): 4096 bytes \/ 4096 bytes<\/strong>
\nDisk identifier: 0x00000000
\n<\/code>
\nDevice Boot Start End Blocks Id System
\n\/dev\/sdc1 2048<\/strong> 1953525134 976761543+ 83 Linux<\/code><\/p>\n

I’ve highlighted the key differences in bold, you can see the physical sector size is 4096 vs. 512 and that is the reason for the different start position. Ok, diversion over – let’s actually follow the wiki<\/a> and get this drive added to the RAID array.<\/p>\n

Start by looking at what we have:<\/p>\n

$ cat \/proc\/mdstat
\nPersonalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10]
\nmd_d3 : active raid5 sdf1[1] sdd1[0] sdb1[2]
\n1953519872 blocks level 5, 64k chunk, algorithm 2 [3\/3] [UUU]<\/code><\/p>\n

So, my RAID5 array is \/dev\/md_d3<\/code>, and I know my new drive is \/dev\/sdc1<\/code> after my parted\/fdisk adventure above.<\/p>\n

$ sudo mdadm --add \/dev\/md_d3 \/dev\/sdc1<\/code><\/p>\n

Now we look at mdstat again and it shows we have a spare. This is honestly what I should have at least done with the drive immediately after installing it – having a spare lets the RAID array fail over to the spare drive with no administrator intervention.<\/p>\n

$ cat \/proc\/mdstat
\nPersonalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10]
\nmd_d3 : active raid5 sdc1[3](S) sdf1[1] sdd1[0] sdb1[2]
\n1953519872 blocks level 5, 64k chunk, algorithm 2 [3\/3] [UUU]<\/code><\/p>\n

Next we grow the array across the new device<\/p>\n

$ sudo mdadm --grow --raid-devices=4 \/dev\/md_d3<\/code><\/p>\n

You can peek at \/proc\/mdstat<\/code> from time to time (or use the watch<\/a> command) to monitor progress. This may take a while.<\/p>\n

Once this is done, don’t forget to modify \/etc\/mdadm\/mdadm.conf<\/code> as per the wiki<\/a>: “To make mdadm find your array edit \/etc\/mdadm.conf and correct the num-devices information of your Array”<\/em><\/p>\n

At this point we now have our data spread across more drives, but don’t have a larger volume. We need to resize the volume to take advantage of the new space. It’s recommended you do the resize with the RAID5 volume unmounted (offline). I set about to do this and hit problems unmounting the volume: this turned out to be samba<\/a> holding on to the volume, turning that service off fixed things.<\/p>\n

Then I hit a show stopper, the resize2fs<\/a> command failed:<\/p>\n

$ sudo resize2fs -p \/dev\/md_d3
\nresize2fs 1.42 (29-Nov-2011)
\nresize2fs: Device or resource busy while trying to open \/dev\/md_d3
\nCouldn't find valid filesystem superblock.<\/code><\/p>\n

Huh? This is something I’ll one day sort out I suppose, but it really beats me what is going on here. You can resize RAID5 while it’s online too, it’s slower and a bit scarier, but it works.<\/p>\n

$ sudo resize2fs \/dev\/md_d3
\nresize2fs 1.42 (29-Nov-2011)
\nFilesystem at \/dev\/md_d3 is mounted on \/stuff; on-line resizing required
\nold_desc_blocks = 117, new_desc_blocks = 175
\nPerforming an on-line resize of \/dev\/md_d3 to 732569952 (4k) blocks.<\/code><\/p>\n

This was followed by a few moments of terror as I realized that I was doing this over a SSH connection – what if the connection is lost?<\/a>\u00a0Next time I’ll use screen, or nohup the process.<\/p>\n

It was neat to watch the free space on the drive creep upwards. It was running at about 1Gb every 2 seconds. Once this finishes, you’re done. My RAID volume went from 1.9T to 2.8T with the new drive.<\/p>\n","protected":false},"excerpt":{"rendered":"

Some time ago I migrated from a RAID1 setup to RAID5, this was on the minimum 3 drives. At some point this summer I spotted a good deal on a matching 1TB drive to what I had in my array and bought it. My purchase sat in my desk drawer for a month (or two) … Continue reading “Add drive to RAID5 on Ubuntu”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,12],"tags":[],"class_list":["post-1303","post","type-post","status-publish","format-standard","hentry","category-computing","category-how-to"],"_links":{"self":[{"href":"https:\/\/lowtek.ca\/roo\/wp-json\/wp\/v2\/posts\/1303","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lowtek.ca\/roo\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lowtek.ca\/roo\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lowtek.ca\/roo\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/lowtek.ca\/roo\/wp-json\/wp\/v2\/comments?post=1303"}],"version-history":[{"count":2,"href":"https:\/\/lowtek.ca\/roo\/wp-json\/wp\/v2\/posts\/1303\/revisions"}],"predecessor-version":[{"id":1305,"href":"https:\/\/lowtek.ca\/roo\/wp-json\/wp\/v2\/posts\/1303\/revisions\/1305"}],"wp:attachment":[{"href":"https:\/\/lowtek.ca\/roo\/wp-json\/wp\/v2\/media?parent=1303"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lowtek.ca\/roo\/wp-json\/wp\/v2\/categories?post=1303"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lowtek.ca\/roo\/wp-json\/wp\/v2\/tags?post=1303"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}