Quick and dirty linux software RAID5

I recently migrated my network storage server, running Ubuntu Linux incidentally, to a software RAID 5 configuration. RAID level 5 requires at least 3 harddrives; giving you in total N-1 storage, where N is the number of disks in the array. For my deployment, I used 3 120GB drives (mixing IDE and SATA, Linux is indifferent here) giving me effectively 240GB. The missing 120GB is the parity information that allows transparent data recovery when 1 disk in the array fails. I’ve glossed over a lot of the details regarding RAID 5 but a quick google should fill in the blanks if need be.

Here is how I accomplished this under Ubuntu Dapper.

Install mdadm using apt-get, or your favorite package manager. mdadm is the tool used to administer Linux md device arrays (software RAID).
Partition your drives as appropriate using cfdisk /dev/hd<?>. Ensure that all partitions are of equal size and the partition type is set to Linux raid autodetect.
Using mdadm create your RAID-5 device:
- mdadm --create /dev/md0 --chunk=64 --level=raid5 --raid-devices=3 /dev/hdc1 /dev/hde1 /dev/sda1; substituting for your own /dev/ devices.
Good Good.
- # cat /proc/mdstat
  Personalities : [raid5]
  md0 : active raid5 sda1[0] hdc1[1] hde1[2]
  234371968 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
Format your newly created /dev/md0 device.
- mkfs.ext3 /dev/md0
Create a mount point for your RAID-5 array and add it to /etc/fstab so it is mounted automagically on the next reboot.
- /dev/md0 /storage ext3 defaults 0 1
Getting detailed info about your array.
- # mdadm --detail /dev/md0
  /dev/md0:
  Version : 00.90.03
  Creation Time : Sat Apr 1 14:01:39 2006
  Raid Level : raid5
  Array Size : 234371968 (223.51 GiB 240.00 GB)
  Device Size : 117185984 (111.76 GiB 120.00 GB)
  Raid Devices : 3
  Total Devices : 3
  Preferred Minor : 0
  Persistence : Superblock is persistentUpdate Time : Tue Apr 18 07:39:21 2006
  State : clean
  Active Devices : 3
  Working Devices : 3
  Failed Devices : 0
  Spare Devices : 0Layout : left-symmetric
  Chunk Size : 64K
  
  UUID : 51dec40b:2d6fd4cc:a9e0addc:fe5303d4
  Events : 0.176640
  
  Number Major Minor RaidDevice State
  0 8 1 0 active sync /dev/sda1
  1 22 1 1 active sync /dev/hdc1
  2 33 1 2 active sync /dev/hde1
Managing the RAID device.
- Setting a disk faulty/failed:
  # mdadm --fail /dev/md0 /dev/hdc1
- Removing a faulty disk from an array:
  # mdadm --remove /dev/md0 /dev/hdc1
- Clearing any previous raid info on a disk:
  # mdadm --zero-superblock /dev/hdc1
- Adding a disk to an array:
  # mdadm --add /dev/md0 /dev/hdc1

RAID can be an intimadating piece of technology but it’s benefits are plainly obvious. Hopefully this framework howto is enough to ease your fears.

16 Comments

Tate October 3, 2006

I found this guide very quick and easy. Thanks very much for taking the time to write it up
Tommi April 11, 2007

I Have to try this with Ubuntu 6.06 LTS server. I have 4 x 250Gt ATA disk and 160Gt install disk for ubuntu.
Mad March 13, 2008

Have just done this on Gutsy (Ubuntu 7.10 64bit). Worked fine, thanks a lot. As a tip, I used System->Administration->Synaptic Package Manager to get the mdadm command installed.
Ivan Lawrence March 26, 2008

Does anyone know how to set up the server to email (or alert another way) for disk failures or array degradation? I have read a few other tutorials but none of them cover proactive disk monitoring.
Thanks,
-Ivan
Ivan Lawrence March 26, 2008

Found it! http://linux-raid.osdl.org/index.php/Detecting%2C_querying_and_testing#Monitoring_RAID_arrays
Adam March 26, 2008

Thanks Ivan! I should have probably included something like this.
Bando Bando May 7, 2008

Guide worked great except my system wouldn’t startup on reboot. I had to use a Recovery Disk to edit out the line in /etc/fstab just so it would boot. It seems there’s one step missing, at least it was required for Fedora 8 x86_64. Anyways. Before rebooting you’ll probably want to:
mdadm –detail –scan –verbose > /etc/mdadm.conf
if you want your system to startup. Like I said could just be a Fedora thing, but I’m betting otherwise. I found this here:
http://www.linuxhomenetworking.com/wiki/index.php/Quick_HOWTO_:_Ch26_:_Linux_Software_RAID
I also used fdisk instead of cfdisk.
Adam Myers May 7, 2008

Hi Bando,

That step doesn’t appear to be necessary in Ubuntu – I don’t recall doing anything other than the above. I’ll be doing another raid5 setup in the next week or so with Hardy Heron. I’ll update my findings then.

Anyways, thanks for the comment!
Paradexes July 5, 2008

Great howto easiest one I have found. Hopefully someone can take this and GUIfy it for new users.

FYI small typos need fixing

–create*
–raid*
–level*
–chunk*

One dash wont work. Still kick ass article otherwise. Works great in Ubuntu 8.04 and 7.10.
Adam Myers July 5, 2008

@Paradexes

Thanks, yeah I’ve noticed that but for some reason my wordpress theme gunks out the double dash.

Glad it proved useful nonetheless!

Edit: I fixed the double dash problem in WordPress
Brian May 21, 2009

AWESOME! Simple, quick and dirty HOW-TO. Thanks so much.
Mad August 28, 2009

This might be the same problem Bando Bando experienced above.
I’ve just installed 4 x 1TB in raid5 in Ubuntu 9.04. After reboot – no md0??? I tried mdadm –assemble –scan, and got only 3 disks. Ok, fine, I’ll add the forth then, I thought. I got device or resource busy. Ok, reboot again, then I only have 2 disks, and array can not be assembled of course. I tried a lot of things like clearing the superblocks, re-partioning etc, same problem still, loses one disk each reboot.
In /etc/mdadm/mdadm.conf it says
DEVICE partitions
CREATE owner=root group=disk mode=0660 auto=yes
HOMEHOST
I replaced these lines with
DEVICE /dev/sd??
ARRAY /dev/md0 UUID=1b0816d7:c82651a0:67e26e6e:a4449dba
The UUID is of course mine, you will find yours using mdadm –detail.
Now it works, I don’t know why. Maybe that I on one disk have a small partition with Ubuntu, and on another disk a swap partition, each 10GB, only then this problem occurs? For example I have another computer where I have 8 dedicated disks in raid5, ie Ubuntu installed on a ninth disk, no changes done in mdadm.conf and despite that everything works fine. Hope this can be of help.
Mad May 10, 2010

I did it again, now with Ubuntu 10.04 64-bit!
I use ext4, ie “sudo mkfs.ext4 /dev/md0″, with success. Same change in fstab of course, ie “/dev/md0 /storage ext4 defaults 0 1″.
Dont forget, you need to create a directory, ie “sudo mkdir /storage”, before you can mount anything to it.
Same as for 9.04 mdadm.conf needs to be to be changed for 10.04 as described above regarding DEVICE and UUID.
Again, thanks Adam for a great howto.