This chapter is from the book
This chapter is from the book
This chapter is from the book
Understanding RAID
RAID (Redundant Array of Independent Disks) allows an administrator to form an array of several hard drives into one logical drive recognized as one drive by the operating system. It also spreads the data stored over the array of drives to decrease disk access time and accomplish data redundancy. The data redundancy can be used to recover data should one of the hard drives in the array crash.
There are two types of RAID: hardware RAID and software RAID. Hardware RAID is implemented through the disk controller for the system. Instructions for configuring hardware RAID differ from controller to controller, so refer to the manual for your disk controller for instructions. Software RAID is implemented through the operating system and does use some processor and memory resources, although some software RAID implementations can produce faster disk access times than hardware RAID.
During installation, it is possible to configure software RAID as discussed in Chapter 1. This section explains the different RAID levels available with software RAID so you can decide which level is best for you. Software RAID allows for RAID levels 0, 1, 5, and 6.
RAID level 0, or striping, means that data is written across all hard drives in the array to accomplish the fast disk performance. No redundancy is used, so the size of the logical RAID drive is equal to the size of all the hard drives in the array. Because there is no redundancy, recovering data from a hard drive crash is not possible through RAID.
RAID level 1, or mirroring, means that all data is written to each disk in the array, accomplishing redundancy. The data is “mirrored” on a second drive. This allows for easy recovery should a disk fail. However, it does mean that, for example, if there are two disks in the array, the size for the logical disk is size of the smaller of the two disks because data must be mirrored to the second disk.
RAID level 5 combines striping and parity. Data is written across all disks as in RAID 0, but parity data is also written to one of the disks. Should a hard drive failure occur, this parity data can be used to recover the data from the failed drive, including while the data is being accessed and the drive is still missing from the array.
RAID level 6 is RAID level 5 with dual parity. Data is written across all disks as in RAID 5, but two sets of parity data is calculated. Performance is slightly worse than RAID 5 because the extra parity data must be calculated and written to disk. RAID 5 allows for recovery using the parity data if only one drive in the array fails. Because of the dual parity, RAID 6 allows for recovery from the failure of up to two drives in the array.
Setting Up RAID Devices
For best results, software RAID should be configured during installation, but it can be configured after installation if necessary. To set up software RAID devices after installation, install the mdadm software package. Refer to Chapter 3 for instructions on installing packages. This section provides an overview of post-installation software RAID configuration. It shows you how to create a RAID array and then move the data from the existing filesystem onto it. Be sure to test the process on a test system before attempting it on a production system.
Before starting the conversion, add the appropriate number of hard drives with the proper sizes for the RAID level. For example, two partitions are needed for RAID 1 (mirroring) and at least three partitions are needed for RAID 5. To use all the benefits of RAID, each partition in a RAID device should be on separate hard drives so each member of the RAID device can be written to at the same time and there is redundancy across separate hard drives should one fail.
It is possible to configure a RAID array with a missing partition so that the data on the existing partition can be copied to the degraded array. The existing partition is reconfigured as a RAID partition and then added to the RAID array to complete it. However, the process for doing so is more complicated and not recommended because it is easier to lose the existing data. It is recommended that new drives be used to set up the RAID device and for the existing data to then be copied to the new RAID device.
When creating partitions to use for the RAID device, make sure they are of type Linux raid auto. In fdisk, this is partition id fd. After creating the partitions for the RAID device, use the following syntax as the root user to create the RAID device:
mdadm --create /dev/mdX --level=<num> --raid-devices=<num> <device list>
The progress of the device creation can be monitored with the following command as root:
tail -f /proc/mdstat
For example, to create a RAID level 1 device /dev/md0 from three partitions, use the following command:
mdadm --create /dev/md0 --level=1 --raid-devices=3 /dev/sda5 /dev/sda6 /dev/sda7
The command cat /proc/mdstat should show output similar to Listing 7.7.
Listing 7.7. Creating a RAID Array
Personalities : [raid0] [raid1]
md0 : active raid1 sda7[2] sda6[1] sda5[0]
10241280 blocks [3/3] [UUU]
[>....................] resync = 0.0% (8192/10241280) finish=62.3min
speed=2730K/sec
unused devices: <none>
The RAID device /dev/md0 is created. Next, create a filesystem on it. To create an ext3 filesystem, execute the following as root:
mke2fs -j /dev/md0
If the new RAID device is to be used as the swap partition, use the following command as root instead:
mkswap /dev/md0
Copy any data over to the new device and be sure to change all references to the old partition to the new RAID device, including /etc/fstab and /etc/grub.conf. It is recommended that the /boot and the / filesystems remain on their original filesystems to ensure the system can still boot after added the RAID devices. Partitions such as /home will benefit from RAID more because data on it changes frequently.
Adding and Failing RAID Partitions
To add a partition to a RAID device, execute the following as root after creating the partition of type Linux raid auto (fd in fdisk):
mdadm /dev/mdX -a <device list>
To add /dev/sda8 to the /dev/md0 RAID device created in the previous section:
mdadm /dev/md0 -a /dev/sda8
Listing 7.8 shows the output from cat /proc/mdstat. The /dev/sda8 partition is now a spare partition in the RAID array.
Listing 7.8. Adding a Spare Partition
Personalities : [raid0] [raid1]
md0 : active raid1 sda8[3](S) sda7[2] sda6[1] sda5[0]
10241280 blocks [3/3] [UUU]
[>....................] resync = 0.6% (66560/10241280) finish=84.0min
speed=2016K/sec
unused devices: <none>
If a partition in the array fails, use the following to remove it from the array and rebuild the array using the spare partition already added:
mdadm /dev/mdX -f <failed device>
For example, to fail /dev/sda5 from /dev/md0 and replace it with the spare (assuming the spare has already been added):
mdadm /dev/md0 -f /dev/sda5
To verify that the device has been failed and that the rebuild has been complete and was successful, monitor the /proc/mdstat file (output shown in Listing 7.9):
tail -f /proc/mdstat
Notice that /dev/sda5 is now failed and that /dev/sda8 has changed from a spare to an active partition in the RAID array.
Listing 7.9. Failing a Partition and Replacing with a Spare
Personalities : [raid0] [raid1]
md0 : active raid1 sda8[3] sda7[2] sda6[1] sda5[4](F)
10241280 blocks [3/2] [_UU]
[>....................] recovery = 0.2% (30528/10241280) finish=11.1min
speed=15264K/sec
unused devices: <none>
Monitoring RAID Devices
The following commands are useful for monitoring RAID devices:
- cat /proc/mdstat: Shows the status of the RAID devices and the status of any actions being performed on them such as adding a new member or rebuilding the array.
mdadm --query /dev/mdX: Displays basic data about the device such as size and number of spares such as:
/dev/md0: 9.77GiB raid1 3 devices, 1 spare.
Add the --detail option to display more data ( mdadm --query --detail /dev/mdX):
/dev/md0: Version : 00.90.03 Creation Time : Mon Dec 18 07:39:05 2006 Raid Level : raid1 Array Size : 10241280 (9.77 GiB 10.49 GB) Device Size : 10241280 (9.77 GiB 10.49 GB) Raid Devices : 3 Total Devices : 4 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Mon Dec 18 07:40:01 2006 State : clean, degraded, recovering Active Devices : 2 Working Devices : 3 Failed Devices : 1 Spare Devices : 1 Rebuild Status : 49% complete UUID : be623775:3e4ed7d6:c133873d:fbd771aa Events : 0.5 Number Major Minor RaidDevice State 3 8 8 0 spare rebuilding /dev/sda8 1 8 6 1 active sync /dev/sda6 2 8 7 2 active sync /dev/sda7 4 8 5 - faulty spare /dev/sda5- mdadm --examine <partition>: Displays detailed data about a component of a RAID array such as RAID level, total number of devices, number of working
devices, and number of failed devices. For example, the output of mdadm --examine /dev/sda6 shows the following:
/dev/sda6: Magic : a92b4efc Version : 00.90.00 UUID : be623775:3e4ed7d6:c133873d:fbd771aa Creation Time : Mon Dec 18 07:39:05 2006 Raid Level : raid1 Device Size : 10241280 (9.77 GiB 10.49 GB) Array Size : 10241280 (9.77 GiB 10.49 GB) Raid Devices : 3 Total Devices : 4 Preferred Minor : 0 Update Time : Mon Dec 18 07:40:01 2006 State : active Active Devices : 2 Working Devices : 3 Failed Devices : 0 Spare Devices : 1 Checksum : ee90b526 - correct Events : 0.5 Number Major Minor RaidDevice State this 1 8 6 1 active sync /dev/sda6 0 0 0 0 0 removed 1 1 8 6 1 active sync /dev/sda6 2 2 8 7 2 active sync /dev/sda7 3 3 8 8 3 spare /dev/sda8
Using MD Multipath
The hard drives in a system are connected to the rest of the system hardware via a disk controller. If the controller fails, the system can no longer communicate with the drives connected to it. However, some systems offer multipath disk access in which more than one controller is connected to the disks. If the active controller fails, a spare one replaces it, allowing continued access to the storage attached.
An example usage of MD Multipath is when a system is connected to a storage area network (SAN) via Fiber Channel Protocol or Cards. The multipath device can represent one interface that connects to the SAN using multiple physical cables. If one or more of the physical connections stops working or gets disconnected, the other physical cables are still active, and the storage is still accessible.
The Linux kernel offers Multiple Device (MD) Multipathing via its software RAID feature. MD Multipathing allows a device to be set up with multiple spares so that if the active device fails, I/O requests do not fail. If the active partition fails, the kernel activates one of the spare partitions as the active one.
To set up an MD Multipath device:
mdadm --create /dev/mdX --level=multipath --raid-devices=<num> <device list>
For example, use the following to set up /dev/md0 with three drives, two of which become spares:
mdadm --create /dev/md0 --level=multipath --raid-devices=3 /dev/sda1 /dev/sdc1 /dev/sdd1
The kernel monitors the failure of the partition and activates a spare when it fails. However, the mdmpd daemon from the mdadm RPM package must be running to automatically add a failed partition back to the array when it becomes available again.