I wanted to build a Xen system with failover so that i can restart the systems with minimal loss of data on my webservers.

Using a SAN or NAS is great but is still a single point of failure unless you pay the big bucks for a complicated network setup, the solution is to have a network RAID 1 using DRBD.

The disk layered structure designed and built is as the following and was done with DRBD ONLY on the Xen shared data volume and not the hosts root file system.

 

 

 LV(xen000)  LV(xen001)
 VG(lvm1)
 DRBD0
 MD3
SDB
SDC

 

NOTE: Part of this document is taken from DTC-Xen instructions and should be taken from their site as they update it frequently and I have my copy for my own references only.

> cd /usr/src/

> apt-get install mercurial

> wget http://bits.xensource.com/oss-xen/release/3.2.1/xen-3.2.1.tar.gz
> hg clone http://xenbits.xensource.com/linux-2.6.18-xen.hg
> wget http://dtcsupport.gplhost.com/xen-patches-2008-06-01.tar.gz

> tar xvfz xen-patches-2008-06-01.tar.gz
> tar xvfz xen-3.2.1.tar.gz

> apt-get install make gcc libc6-dev zlib1g-dev python python-dev python-twisted bridge-utils iproute libcurl3 libcurl3-dev bzip2 module-init-tools transfig tgif libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg62-dev bcc bin86 gawk pciutils-dev lvm2 flex tetex-bin gs-common

> cd /usr/src/linux-2.6.18-xen.hg/
> /usr/src/xen-patches/patch-xen-hg

> cd /usr/src/xen-3.2.1
> make world KERNELS="linux-2.6-xen0 linux-2.6-xenU"
# Copy .configs for both (if already preconfigured) and `make oldconfig`, ....
> scp config.build-linux-2.6.18-xen0_x86_64 \
root@10.10.0.40 This e-mail address is being protected from spam bots, you need JavaScript enabled to view it :/usr/src/xen-3.2.1/build-linux-2.6.18-xen0_x86_64/.config

> scp config.build-linux-2.6.18-xenU_x86_64 \
root@10.10.0.40 This e-mail address is being protected from spam bots, you need JavaScript enabled to view it :/usr/src/xen-3.2.1/build-linux-2.6.18-xenU_x86_64/.config

# otherwise use `make menuconfig` and configure it yourself.

# NOTE: Make sure "Drivers -> Connector - unified userspace <-> kernelspace linker " is enabled for drbd compilin to work.


# Now finally make the kernels.

> make dist KERNELS="linux-2.6-xen0 linux-2.6-xenU"
> make install KERNELS="linux-2.6-xen0 linux-2.6-xenU"

> vi /boot/grub/menu.lst
# Add the following lines above the  "## ## End Default Options ##" line in your menu list.
title           Xen 3.0 / XenLinux 2.6
kernel          /boot/xen-3.2.1.gz dom0_mem=393216
module          /boot/vmlinuz-2.6.18.8-xen0 root=/dev/md0 ro console=tty0 max_cpus=1
#  (NOTE:  Remove /boot if /boot has its own partition from root and change root=/dev/md0 to what ever root is on )

> update-rc.d xend defaults 20 21
> update-rc.d xendomains defaults 21 20

# Cross fingers and reboot
> reboot

> vi /etc/apt/source.list
# Add the following
deb ftp://ftp.gplhost.com/debian stable main # US main ftp
deb ftp://ftp.gplhost.sg/debian stable main # Singapore mirror (for asian users)
deb ftp://ftp.gplhost.fr/debian stable main # Paris mirror (for european users)

> apt-get update

> wget http://oss.linbit.com/drbd/8.2/drbd-8.2.6.tar.gz
> tar xvfz drbd-8.2.6.tar.gz
> cd drbd-8.2.6
> make install

# Edit /etc/drbd.conf and /etc/init.d/drbd restart
> drbdadm create-md xenlvm1
> drbdadm -- --overwrite-data-of-peer primary all
> drbdadm primary all

> vi /etc/lvm/lvm.conf
# Change the filter line to exclude any MD devices or SCSI disks but allow/read DRBD devices
filter = [ "r|/dev/cdrom|","r|/dev/sda*|", "r:/dev/md*|", "r|/dev/sdb*|", "a|/dev/drbd*|" ]

> /etc/init.d/lvm restart

> pvcreate /dev/drbd0
#  Physical volume "/dev/drbd0" successfully created

> vgcreate lvm1 /dev/drbd0
#  Volume group "lvm1" successfully created

vgcreate lvm1 /dev/drbd0
# NOTE: if you get something similar as below, your filter is not correctly setup and my example above isn't enough.
#  Found duplicate PV fpCn3ra1k796umKCOIynSB8eTwdBpu4J: using /dev/hda4 not /dev/drbd0
#  Found duplicate PV fpCn3ra1k796umKCOIynSB8eTwdBpu4J: using /dev/drbd0 not /dev/hda4
#  Found duplicate PV fpCn3ra1k796umKCOIynSB8eTwdBpu4J: using /dev/hda4 not /dev/drbd0
#  Found duplicate PV fpCn3ra1k796umKCOIynSB8eTwdBpu4J: using /dev/drbd0 not /dev/hda4
#  Volume group "lvm1" successfully created

> apt-get install smartmontools
> vi /etc/smartd.conf
# Remove the auto detect part
/dev/sda -d ata -H -l error -l selftest -f -m admin@example.com This e-mail address is being protected from spam bots, you need JavaScript enabled to view it
/dev/sdb -d ata -H -l error -l selftest -f -m admin@example.com This e-mail address is being protected from spam bots, you need JavaScript enabled to view it

# DONE. your DTC-Xen with DRBD as a network RAID1 is now complete.