WareWulf How-To: SysAdmins

Warewulf is a scalable systems management suite originally developed to manage large high-performance Linux clusters. Focused on general scalable systems management, it includes a framework for system configuration, management, provisioning/installation, monitoring, event notification, and more via a modular plugin architecture. Install the components and features you need or leverage the existing system configurations stored within Warewulf to create custom solutions to meet your particular needs.

Warewulf is a flexible solution that has proven itself to be scalable and easy to use.

1. Installation

Reference: http://warewulf.lbl.gov/trac/wiki/Recipes/Installation

YUM Installation Method

$ wget -O /etc/yum.repos.d/warewulf-rhel6.repo http://warewulf.lbl.gov/downloads/repo/warewulf-rhel6.repo
$ yum install warewulf-common warewulf-provision warewulf-provision-server  warewulf-vnfs tcpdump \
              mysql mysql-server

1.1. Create Database

Create DB, Modify Configs

$ service mysqld start
$ mysqladmin create warewulf
$ mysql warewulf < /usr/share/warewulf/setup.sql
$ mysqladmin -u root password superSecurePasswordGoesHere
$ vim /etc/warewulf/database.conf

1.2. Create Bootstrap

Reference: http://warewulf.lbl.gov/trac/wiki/Recipes/Provisioning/RHEL

Command

$ wwbootstrap `uname -r`

1.3. Create VNFS

Reference: http://warewulf.lbl.gov/trac/wiki/Recipes/Provisioning/RHEL

VNFS Creation

$ wwmkchroot sl-6 /var/chroots/sl-6-stateless
$ yum --installroot=/var/chroots/sl-6-stateless update
$ echo "yes" | wwvnfs --root /var/chroots/sl-6-stateless

1.4. Setup new Node (Manually)

We are assuming this is going to be a state less install for WareWulf. Reference: http://warewulf.lbl.gov/trac/wiki/Provision

Command

$ wwsh node new a64-999 --netdev=eth0 --hwaddr=00:00:00:00:00:00 \
  --ipaddr=10.255.0.0 -G 10.255.78.1 -M 255.255.255.0 \
  --groups=bduc,long,adc -d bduc
$ provision set --lookup groups bduc --vnfs=sl-6-stateless \
  --bootstrap=2.6.32-220.4.1.el6.x86_64

1.5. Modify or Replace an existing node

Hardware dies. So we ocassionally need to replace an old node with a new one. WW makes it very easy to do this. Since all the WW config info lives in the WW MySQL DB and is accessed via the wwsh interface, we typcially only need to change the MAC address and reboot.

First, on the new node, obtain the primary MAC adddress, typically via the BIOS altho some vendors will intelligently put it on the case as well (not Dell). Also via BIOS, set the appro interface to PXE-boot. Then modify the node config via wwsh.

[root@warewulf ~]# wwsh
Warewulf> node set a64-128 --hwaddr=00:1a:a0:0a:01:5c

answer yes and you’re done. WW will then migrate the changes to the DHCP config file and restart the dhcpd service. Save the BIOS changes if any, reboot the node, and it should boot the default vnfs.

1.6. Modifications of the VNFS

Often, you’ll want to change the VNFS image (change execution, add packages, add a symlink, etc). This is done on the chrooted filesystem, on our system at /var/chroots which in the current state has only 2 chroot environments:

sl-6-statefull
sl-6-stateless

Some file operations can be done on the overall filesystem (editing config files, editing/deleting files, etc); some must be done in the chrooted env: ie

[root@warewulf ~]# cd /var/chroots/
[root@warewulf chroots]# chroot sl-6-stateless
[root@warewulf /]#  <--- now in the chroot

when you’ve finished modifying the chroot filesystem, you then have to re-compress and process it into a transmittable VNFS. This is done like so:

# have to answer 'yes' to overwrite the existing one
   wwvnfs --root /var/chroots/sl-6-stateless
# or in one shot..
   echo "yes" | wwvnfs --root /var/chroots/sl-6-stateless

1.7. Setup new Node (Automatically)

We are assuming this is going to be a state less install for WareWulf. Reference: http://warewulf.lbl.gov/trac/wiki/Provision

Command

$ wwnodescan --netdev=eth0 --ipaddr=10.255.0.0 --netmask=255.255.0.0 \
  --vnfs=rhel-6 --bootstrap=2.6.32-71.18.2.el6.x86_64 a64-[900-999]

2. What If

2.1. WareWulf Master Goes Down?

Each compute node will continue to run normally and will remain online and within the SGE pool. If the compute nodes are rebooted while the WareWulf master is still down, they will not come back online.

2.2. Compute Node goes Down?

Reboot. Assuming a stateless install, on reboot, the machine will load the OS into RAMDISK and it will act as a fresh install. If stateful, it will boot from hard drive.

2.3. I want to re-image a node?

If state less, reboot the node. Stateless nodes get "re-imaged" upon every reboot as the OS lives in RAMDISK. If the node is stateful, run the command echo "yes" | wwsh object modify -s bootlocal=0 nodeNameHere which will have the server boot the from PXE and not local disk. This will format the drive. Upon reboot, set bootlocal back to 1.

3. Resources

The WareWulf community provides quite a few resources for new users on how to use and setup WareWulf. For your reference, they are listed below.

Table 1. Resources
Name	Link
WareWulf	http://warewulf.lbl.gov/
WareWulf WiKi	http://warewulf.lbl.gov/trac/wiki/Documentation
WareWulf Terminology	http://warewulf.lbl.gov/trac/wiki/Terms
WareWulf Architecture	http://warewulf.lbl.gov/trac/wiki/Architecture
HPC Magazine #1	Warewulf Cluster Manager – Part 1: Master and Compute Nodes
HPC Magazine #2	Warewulf Cluster Manager – Part 2: Completing the environment
HPC Magazine #3	Warewulf Cluster Manager – Part 3: Development and Run Time
HPC Magazine #4	Warewulf Cluster Manager – Part 4: Administration and Monitoring

(thanks to Nick Santucci for providing corrected links)