HOWTO Improve availability with Software RAID
From Consultancy.EdVoncken.NET
My main home-server uses Software RAID-5 using 4x 500GB SATA disks for improved availability. It has proven to be very reliable over the years. This does not mean you can do without backups though!
Contents |
Key points:
- RAID vs. Backup
- Repeat after me: "RAID Is Not Backup!" - if you accidentally delete a file, it's gone from your RAID drives but probably still on your backup disk/tape somewhere. RAID is for availability, backup is for disaster recovery.
- Software-RAID vs. Hardware-RAID
- The advantage of Software-RAID over Hardware-RAID is that you don't need specialized controllers; any mainboard with plenty of SATA ports will do. With Hardware-RAID, you need to keep a spare controller board in case you encounter a controller failure. The performance of Software-RAID is more than sufficient for most applications; in fact, Software-RAID outperforms most of the commonly available Hardware-RAID adapters.
Availability goals
Our goal is to protect both the OS and the applications/data against a single disk fault. We will use RAID (Redundant Array of Inexpensive/Independent Disks) to achieve this goal.
RAID-5 is the most popular choice for normal use; the overhead is 1 disk, so your remaining storage out of n disks is (n-1) times single disk capacity.
Design
Ideally, everything should be stored inside the RAID array. As GRUB (the Linux boot-loader) cannot boot from a RAID-5 array, the /boot partition should be stored on a separate RAID-1 array - GRUB will boot from a RAID-1 array.
This means we will end up with two RAID arrays on the same set of disks:
- /dev/md0
- RAID-1 array, approx. 128MB, across all available disks.
Will hold the /boot partition. - /dev/md1
- RAID-5 array, remaining space on all available disks.
Will hold all other partitions.
This setup should protect against all single-disk failures.
Note: If you are still using PATA (IDE) type harddisks, you should put each disk on its own controller - with a master/slave setup, one failing disk or controller can put 2 drives out of action, resulting in complete loss of availability.
Installation and Configuration
Installation
The Anaconda installer will prompt you to enter the RAID configuration. The RAID arrays will be created during installation, and the GRUB boot-loader will be installed on all drives.
Partitioning
Each disk (/dev/sda through /dev/sdd) has been partitioned as follows:
[root@hal ~]# fdisk -l /dev/sda
Disk /dev/sda: 500.1 GB, 500106780160 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 16 128488+ fd Linux raid autodetect
/dev/sda2 17 60801 488255512+ fd Linux raid autodetect
The same information, but obtained with parted:
[root@hal ~]# parted /dev/sda print Model: ATA WDC WD5000ABYS-0 (scsi) Disk /dev/sda: 500GB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type File system Flags 1 32.3kB 132MB 132MB primary ext3 boot, raid 2 132MB 500GB 500GB primary raid Information: Don't forget to update /etc/fstab, if necessary.
Note that all partition types have been set to 0xFD, for "Linux Raid Autodetect".
Boot loader
The GRUB boot-loader must be installed on all drives to ensure that the system is bootable even in case of a disk failure.
Verify that Grub is installed:
# file -s /dev/sda # file -s /dev/sdb # file -s /dev/sdc # file -s /dev/sdd
The output should be similar to the following:
/dev/sda: x86 boot sector; partition 1: ID=0xfd, active, starthead 1, startsector 63, 256977 sectors;
partition 2: ID=0xfd, starthead 0, startsector 257040, 976511025 sectors, code offset 0x48
Details for /dev/md0
[root@hal ~]# mdadm --detail /dev/md0
/dev/md0:
Version : 0.90
Creation Time : Sun Sep 2 18:57:09 2007
Raid Level : raid1
Array Size : 128384 (125.40 MiB 131.47 MB)
Used Dev Size : 128384 (125.40 MiB 131.47 MB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Sun Jan 31 04:22:06 2010
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
UUID : 7b74288c:bf3fdd67:5bca0d70:470d64d9
Events : 0.36
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
2 8 33 2 active sync /dev/sdc1
3 8 49 3 active sync /dev/sdd1
The RAID-1 array is formatted as ext3, and mounted as /boot.
Details for /dev/md1
LVM + RAID
For maximum flexibility, use Logical Volume Management to distribute storage across the remaining partitions. /dev/md1 will accommodate the OS installation, virtual machine images as well as all user data. Traditional partitioning is not flexible enough for this kind of setup.
TODO: (Work in Progress)
Navigation
- HOWTO Improve availability with Software RAID