Bioinformatics

This page describes how to install and use basic features of the Kerrighed cluster operating system.

Installation

The Kerrighed Installation Manual details how to compile the kernel from source. This is straight forward but it is useful to be aware of a few points.

If you want to use a firewall you will need to enable xtables / iptables in the kernel. Note that there is a problem with experimental SATA drivers in the linux 2.6.20 kernel. These are enabled by default if you let menuconfig create your base configuration file. To avoid the problem (namely kernel panic during the boot process) run 'make defconfig' first to create a working configuration and then load this into menuconfig for editing.

If you are running Debian, try out these debs (experimental) and post feedback comments!

Configuration

Before you run the cluster for the first time, you need to do some configuration.

You can configure Grub to pass kernel parameters session_id and node_id on each node. The former must be the same across all nodes, the latter unique. By example (in /boot/grub/menu.lst);

> kernel    /boot/vmlinuz root=/dev/sda1 ro session_id=1 node_id=1

The configuration file /etc/kerrighed_nodes should contain at least the session id and the nbmin field, specifying the minimum number of nodes to be running the kerrighed module for the cluster to start automatically. Nodes can be specified explicitly by ip address.

sample kerrighed_nodes file:

session = 1
nbmin = 3
192.168.0.103:1:eth1
192.168.0.104:2:eth1
192.168.0.101:3:eth1

Finally, setting the enable option to TRUE in /etc/default/kerrighed allows for the Kerrighed module to load during the boot process.

Running Kerrighed

Use the krgadm tool to start the cluster;

# krgadm cluster start

(Note that the kerrighed module must be running first. You can check for this in /var/log/kern.log. If the module is not loaded, load it with '# /etc/init.d/kerrighed start')

Running top and typing 1 should now display the processors of all nodes in the cluster.

To migrate processes, use krgcapset to set permissions and then use the migrate command;

# krgcapset -k  [pid] --effective +CAN_MIGRATE
# migrate [pid] [node_id]

You can use $$ to retrieve the pid of the shell from which you are running processes. The following allows for sub processes to migrate at the discretion of Kerrighed.

# krgcapset -k $$ --inheritable-effective +CAN_MIGRATE

Note that some processes do not seem to migrate easily. Migration is demonstrable for memory intensive processes called from R but apparently not the processor intensive dhrystone benchmark.