WareWulf How-To: SysAdmins ========================== Adam Brenner , http://www.ics.uci.edu/~aebrenne/ v1.02, 2012-09-25 // WareWulf How-TO (c) 2012 University of California, Irvine // ---------------------------------------------------------------------------- // This document serves as a user manual for WareWulf within the BDUC cluster // for OIT @ UCI. However the commands and concepts can be applied elsewhere // outside the BDUC environment. // // Command: // cd /var/www/html/guides/ // asciidoc -b html5 -a icons -a toc2 -a numbered /home/aebrenne/guides/warewulf.txt // mv /home/aebrenne/guides/warewulf.* . // // asciidoc cheatsheet: http://powerman.name/doc/asciidoc // asciidoc user guide: http://www.methods.co.nz/asciidoc/userguide.html // // @author Adam Brenner // @url http://www.ics.uci.edu/~aebrenne/ // @version 1.0 // addition for hjm additions: //// fileroot="/home/hjm/nacs/hpc/warewulf"; asciidoc -a icons -a toc2 -b html5 -a numbered ${fileroot}.txt; scp ${fileroot}.html ${fileroot}.txt moo:~/public_html/bduc Warewulf is a scalable systems management suite originally developed to manage large high-performance Linux clusters. Focused on general scalable systems management, it includes a framework for system configuration, management, provisioning/installation, monitoring, event notification, and more via a modular plugin architecture. Install the components and features you need or leverage the existing system configurations stored within Warewulf to create custom solutions to meet your particular needs. Warewulf is a flexible solution that has proven itself to be scalable and easy to use. Installation ------------ Reference: http://warewulf.lbl.gov/trac/wiki/Recipes/Installation .YUM Installation Method ------------------------ $ wget -O /etc/yum.repos.d/warewulf-rhel6.repo http://warewulf.lbl.gov/downloads/repo/warewulf-rhel6.repo $ yum install warewulf-common warewulf-provision warewulf-provision-server warewulf-vnfs tcpdump \ mysql mysql-server ------------------------ Create Database ~~~~~~~~~~~~~~~ .Create DB, Modify Configs -------------------------- $ service mysqld start $ mysqladmin create warewulf $ mysql warewulf < /usr/share/warewulf/setup.sql $ mysqladmin -u root password superSecurePasswordGoesHere $ vim /etc/warewulf/database.conf -------------------------- Create Bootstrap ~~~~~~~~~~~~~~~~ Reference: http://warewulf.lbl.gov/trac/wiki/Recipes/Provisioning/RHEL .Command -------- $ wwbootstrap `uname -r` -------- Create VNFS ~~~~~~~~~~~ Reference: http://warewulf.lbl.gov/trac/wiki/Recipes/Provisioning/RHEL .VNFS Creation -------------- $ wwmkchroot sl-6 /var/chroots/sl-6-stateless $ yum --installroot=/var/chroots/sl-6-stateless update $ echo "yes" | wwvnfs --root /var/chroots/sl-6-stateless -------------- Setup new Node (Manually) ~~~~~~~~~~~~~~~~~~~~~~~~~ We are assuming this is going to be a state *less* install for WareWulf. Reference: http://warewulf.lbl.gov/trac/wiki/Provision .Command -------- $ wwsh node new a64-999 --netdev=eth0 --hwaddr=00:00:00:00:00:00 \ --ipaddr=10.255.0.0 -G 10.255.78.1 -M 255.255.255.0 \ --groups=bduc,long,adc -d bduc $ provision set --lookup groups bduc --vnfs=sl-6-stateless \ --bootstrap=2.6.32-220.4.1.el6.x86_64 -------- Modify or Replace an existing node ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Hardware dies. So we ocassionally need to replace an old node with a new one. WW makes it 'very' easy to do this. Since all the WW config info lives in the WW MySQL DB and is accessed via the 'wwsh' interface, we typcially only need to change the MAC address and reboot. First, on the new node, obtain the primary MAC adddress, typically via the BIOS altho some vendors will intelligently put it on the case as well (not Dell). Also via BIOS, set the appro interface to PXE-boot. Then modify the node config via 'wwsh'. -------------------------------------- [root@warewulf ~]# wwsh Warewulf> node set a64-128 --hwaddr=00:1a:a0:0a:01:5c -------------------------------------- answer 'yes' and you're done. WW will then migrate the changes to the DHCP config file and restart the dhcpd service. Save the BIOS changes if any, reboot the node, and it should boot the default vnfs. Modifications of the VNFS ~~~~~~~~~~~~~~~~~~~~~~~~~ Often, you'll want to change the VNFS image (change execution, add packages, add a symlink, etc). This is done on the chrooted filesystem, on our system at */var/chroots* which in the current state has only 2 chroot environments: - sl-6-statefull - sl-6-stateless Some file operations can be done on the overall filesystem (editing config files, editing/deleting files, etc); some must be done in the chrooted env: ie ------------------------ [root@warewulf ~]# cd /var/chroots/ [root@warewulf chroots]# chroot sl-6-stateless [root@warewulf /]# <--- now in the chroot ------------------------ when you've finished modifying the chroot filesystem, you then have to re-compress and process it into a transmittable VNFS. This is done like so: ------------------------ # have to answer 'yes' to overwrite the existing one wwvnfs --root /var/chroots/sl-6-stateless # or in one shot.. echo "yes" | wwvnfs --root /var/chroots/sl-6-stateless ------------------------ Setup new Node (Automatically) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We are assuming this is going to be a state *less* install for WareWulf. Reference: http://warewulf.lbl.gov/trac/wiki/Provision .Command -------- $ wwnodescan --netdev=eth0 --ipaddr=10.255.0.0 --netmask=255.255.0.0 \ --vnfs=rhel-6 --bootstrap=2.6.32-71.18.2.el6.x86_64 a64-[900-999] -------- What If ------- WareWulf Master Goes Down? ~~~~~~~~~~~~~~~~~~~~~~~~~~ Each compute node will continue to run normally and will remain online and within the SGE pool. If the compute nodes are rebooted while the WareWulf master is still down, they will not come back online. Compute Node goes Down? ~~~~~~~~~~~~~~~~~~~~~~~ Reboot. Assuming a stateless install, on reboot, the machine will load the OS into RAMDISK and it will act as a fresh install. If stateful, it will boot from hard drive. I want to re-image a node? ~~~~~~~~~~~~~~~~~~~~~~~~~~ If state *less*, reboot the node. Stateless nodes get "re-imaged" upon every reboot as the OS lives in RAMDISK. If the node is stateful, run the command +echo "yes" | wwsh object modify -s bootlocal=0 nodeNameHere+ which will have the server boot the from PXE and not local disk. This will format the drive. Upon reboot, set bootlocal back to 1. Resources --------- The WareWulf community provides quite a few resources for new users on how to use and setup WareWulf. For your reference, they are listed below. .Resources [width="100%",options="header"] |=============================================================================== | Name | Link | WareWulf | http://warewulf.lbl.gov/ | WareWulf WiKi | http://warewulf.lbl.gov/trac/wiki/Documentation | WareWulf Terminology | http://warewulf.lbl.gov/trac/wiki/Terms | WareWulf Architecture | http://warewulf.lbl.gov/trac/wiki/Architecture |HPC Magazine #1 | http://goo.gl/Ku6JqF[Warewulf Cluster Manager – Part 1]: + Master and Compute Nodes |HPC Magazine #2 | http://goo.gl/vHGijy[Warewulf Cluster Manager – Part 2]: + Completing the environment |HPC Magazine #3 | http://goo.gl/0egBzu[Warewulf Cluster Manager – Part 3]: + Development and Run Time |HPC Magazine #4 | http://goo.gl/gDNjSV[Warewulf Cluster Manager – Part 4]: + Administration and Monitoring |=============================================================================== (thanks to Nick Santucci for providing corrected links)