Different RAID levels .....!

What are the different RAID levels? Which one is recommended for file server and database server?

A Redundant Array of Independent Drives (or Disks), also known as Redundant Array of Inexpensive Drives (or Disks) (RAID) is an term for data storage schemes that divide and/or replicate data among multiple hard drives. RAID can be designed to provide increased data reliability or increased I/O performance, though one goal may compromise the other.

There are total 10 types of RAID levels:

• RAID level 0
• RAID level RAID level 1
• RAID level 2
• RAID level 3
• RAID level 4
• RAID level 5
• RAID level 6
• RAID level 10
• RAID level 50
• RAID level 0+1

Commonly used RAID levels for UNIX / Linux and Windows server

Following are commonly used RAID levels :

RAID level	Minimum hard disks	Suggested application	Notes
RAID 0 - Striped Set without parity	2 Hard disks	1. Video Production and Editing 2. Image Editing 3. Any application requiring high bandwidth	Provides improved performance and additional storage but no fault tolerance from disk errors or disk failure. Any disk failure destroys the array, which becomes more likely with more disks in the array.
RAID 1 -MirroredSet (2 disks minimum) without parity.	2 Hard disks	1. Office application 2. Financial application 3. Payroll application etc	Provides fault tolerance from disk errors and single disk failure. Increased read performance occurs when using a multi-threaded operating system that supports split seeks, very small performance reduction when writing. Array continues to operate so long as at least one drive is functioning
RAID 5	3 Hard disks	1. File and Application servers 2. Internet Web, E-mail servers 3. Intranet servers	Highest Read data transaction rate, Medium Write data transaction rate, Overall good (aggregate) transfer rate. drive failure requires replacement, but the array is not destroyed by a single drive failure. Upon drive failure, any subsequent reads can be calculated from the distributed parity such that the drive failure is masked from the end user. The array will have data loss in the event of a second drive failure and is vulnerable until the data that was on the failed drive is rebuilt onto a replacement drive
RAID 10 (nested RAID 1+0)	4 Hard disks	1. Database server (such as Oracle / MySQL / MS-SQL) which requiring high performance and fault tolerance	Provides fault tolerance and improved performance but increases complexity.

See also - Understanding RAID and required number of minimum disk in graphical format below.

I had noticed that many new admins and end user get confused with RAID concept. Many think that RAID 0 offers backup protection.

A few points:
a) Redundant array of independent disks (RAID) refers to a data storage scheme using multiple hard drives to share or replicate data among the drives.

b) No matter which RAID array you use backup regularly.

c) For performance always use hardware based RAID controller. If you are tight on budget go for software based RAID. For any production environment I recommend hardware based RAID controller.

c) To make your decision final regarding RAID consider following points:
Your requirements

• Fault tolerant
• Speed vs performance vs overhead
• Your application and required bandwidth
• Final Cost

(click to enlarge image)

How to know the Hardware details in Linux

 Use command called lsdev. It gathers information about your computer's installed hardware from the interrupts, ioports and dma files in the /proc directory, thus giving you a quick overview of which hardware uses what I/O addresses and what IRQ and DMA channels.

This command is very useful while troubleshooting various hardware conflict problems.

Task display : CPU Information

$ cat /proc/cpuinfo

Task: Display Free and used Memory Information

$ free -m

Task: Display partition information

# fdisk -l

Task: Display file system disk space usage

# df -H
OR
# df -h

Task: Estimate file space usage

# du

Task: List PCI devices

# lspci

Task: List USB devices

# lsusb

Task display : More Information

$ lsdev

Usage of tar command over secure ssh session

How do I use tar command over secure ssh session?

The GNU version of the tar archiving utility (and other old version of tar) can be use through network over ssh session. Do not use telnet command, it is insecure. You can use Unix/Linux pipes to create actives. Following command backups /wwwdata directory to dumpserver.nixcraft.in (IP 192.168.1.201) host over ssh session.

The default first SCSI tape drive under Linux is /dev/st0. You can read more about tape drives naming convention used under Linux here.

# tar zcvf - /wwwdata | ssh root@dumpserver.nixcraft.in "cat > /backup/wwwdata.tar.gz"OR# tar zcvf - /wwwdata | ssh root@192.168.1.201 "cat > /backup/wwwdata.tar.gz"

Output:

tar: Removing leading `/' from member names
/wwwdata/
/wwwdata/n/nixcraft.in/
/wwwdata/c/cyberciti.biz/
....
..
...
Password:

You can also use dd command for clarity purpose:
# tar cvzf - /wwwdata | ssh root@192.168.1.201 "dd of=/backup/wwwdata.tar.gz"
It is also possible to dump backup to remote tape device:
# tar cvzf - /wwwdata | ssh root@192.168.1.201 "cat > /dev/nst0"
OR you can use mt to rewind tape and then dump it using cat command:
# tar cvzf - /wwwdata | ssh root@192.168.1.201 $(mt -f /dev/nst0 rewind; cat > /dev/nst0)$
You can restore tar backup over ssh session:# cd /
# ssh root@192.168.1.201 "cat /backup/wwwdata.tar.gz" | tar zxvf -If you wish to use above command in cron job or scripts then consider SSH keys to get rid of the passwords.

Howto to take Tape Backup With mt And tar Command

Magnetic tape is a non-volatile storage medium consisting of a magnetic coating on a thin plastic strip. Nearly all recording tape is of this type, whether used for video, audio storage or general purpose digital data storage using a computer. How do I make backup using tapes under Linux operating systems?

Linux (and other Unixish system) use mt command to control magnetic tape drive operation. You need to use mt command while working with tape drive. It allows you to reading and writing to tape.

The default tape drive under Linux is /dev/st0 (first SCSI tape device name).Following paragraph summaries command you need to use control tape drive for backup/restore purpose.

Rewind tape drive:# mt -f /dev/st0 rewindBackup directory /www and /home with tar command (z - compressed):# tar -czf /dev/st0 /www /homeFind out what block you are at with mt command:# mt -f /dev/st0 tellDisplay list of files on tape drive:# tar -tzf /dev/st0Restore /www directory:# cd /
# mt -f /dev/st0 rewind
# tar -xzf /dev/st0 wwwUnload the tape:# mt -f /dev/st0 offlineDisplay status information about the tape unit:# mt -f /dev/st0 statusErase the tape:# mt -f /dev/st0 eraseYou can go BACKWARD or FORWARD on tape with mt command itself:
(a) Go to end of data:# mt -f /dev/nst0 eod(b) Goto previous record:# mt -f /dev/nst0 bsfm 1(c) Forward record:# mt -f /dev/nst0 fsf 1Replace /dev/st0 with your actual tape drive name.

Linux Tape Backup Example

To backup to multiple tape use the following command (backup /home file system):
# tar -clpMzvf /dev/st0 /home
To compare tape backup, enter:
# tar -dlpMzvf /dev/st0 /home
To restore tape in case of data loss or hard disk failure:
# tar -xlpMzvf /dev/st0 /home
Where,

• d : find differences between archive and file system
• x : extract files from an archive
• l : list the contents of an archive
• p : ignore umask when extracting files
• M : create/list/extract multi-volume archive (multiple tapes)
• z : Compress backup using gzip
• v : verbosely list files processed
• f /dev/st0 : Tape device name
• /home : Backup /home file system

Putting it all tougher

#!/bin/bash
# A UNIX / Linux shell script to backup dirs to tape device like /dev/st0 (linux)
# This script make both full and incremental backups.
# You need at two sets of five  tapes. Label each tape as Mon, Tue, Wed, Thu and Fri.
# You can run script at midnight or early morning each day using cronjons.
# The operator or sys admin can replace the tape every day after the script has done.
# Script must run as root or configure permission via sudo.
# -------------------------------------------------------------------------
# Copyright (c) 1999 Vivek Gite <vivek@nixcraft.com>
# This script is licensed under GNU GPL version 2.0 or above
# -------------------------------------------------------------------------
# This script is part of nixCraft shell script collection (NSSC)
# Visit http://bash.cyberciti.biz/ for more information.
# -------------------------------------------------------------------------
# Last updated on : March-2003 - Added log file support.
# Last updated on : Feb-2007 - Added support for excluding files / dirs.
# -------------------------------------------------------------------------
LOGBASE=/root/backup/log
 
# Backup dirs; do not prefix /
BACKUP_ROOT_DIR="home sales"
 
# Get todays day like Mon, Tue and so on
NOW=$(date +"%a")
 
# Tape devie name
TAPE="/dev/st0"
 
# Exclude file
TAR_ARGS=""
EXCLUDE_CONF=/root/.backup.exclude.conf
 
# Backup Log file
LOGFIILE=$LOGBASE/$NOW.backup.log
 
# Path to binaries
TAR=/bin/tar
MT=/bin/mt
MKDIR=/bin/mkdir
 
# ------------------------------------------------------------------------
# Excluding files when using tar
# Create a file called $EXCLUDE_CONF using a text editor
# Add files matching patterns such as follows (regex allowed):
# home/vivek/iso
# home/vivek/*.cpp~
# ------------------------------------------------------------------------
[ -f $EXCLUDE_CONF ] && TAR_ARGS="-X $EXCLUDE_CONF"
 
#### Custom functions #####
# Make a full backup
full_backup(){
	local old=$(pwd)
	cd /
	$TAR $TAR_ARGS -cvpf $TAPE $BACKUP_ROOT_DIR
	$MT -f $TAPE rewind
	$MT -f $TAPE offline
	cd $old
}
 
# Make a  partial backup
partial_backup(){
	local old=$(pwd)
	cd /
	$TAR $TAR_ARGS -cvpf $TAPE -N "$(date -d '1 day ago')" $BACKUP_ROOT_DIR
	$MT -f $TAPE rewind
	$MT -f $TAPE offline
	cd $old
}
 
# Make sure all dirs exits
verify_backup_dirs(){
	local s=0
	for d in $BACKUP_ROOT_DIR
	do
		if [ ! -d /$d ];
		then
			echo "Error : /$d directory does not exits!"
			s=1
		fi
	done
	# if not; just die
	[ $s -eq 1 ] && exit 1
}
 
#### Main logic ####
 
# Make sure log dir exits
[ ! -d $LOGBASE ] && $MKDIR -p $LOGBASE
 
# Verify dirs
verify_backup_dirs
 
# Okay let us start backup procedure
# If it is monday make a full backup;
# For Tue to Fri make a partial backup
# Weekend no backups
case $NOW in
	Mon)	full_backup;;
	Tue|Wed|Thu|Fri) 	partial_backup;;
	*) ;;
esac > $LOGFIILE 2>&1
 

Customize above shell script as per your needs and setup a cron job to execute it:
@midnight /path/to/tapebackup.sh