CbN is a Linux live distribution that allows one to run standard desktop hardware as an mpi-based computing cluster without modification to the local hard disk or network topology (i.e. cluster nodes can receive their ip address from the existing dhcp server). The distribution is minimal–there is no support for nfs or a desktop environment. The master node takes about 30 mb of ram, and the compute nodes even less. Compile an executable or write a Ruby or Lua script, and send it to the master node, which can distribute it over ssh and run it. The distribution is based on Tiny Core Linux.
This project was initially inspired by the far more feature-rich Pelican HPC, though no code is shared.
The current release (version 0.1) of Cluster by Night is built on Tiny Core Linux's microcore_2.2rc3, runs Open mpi 1.3.3, comes with Clustalw-mpi 0.13 and Ruby 1.9.1, and is distributed as two 14 mb live-booting iso images.
Cluster by Night Screencast from Dept. Dirigible Flightcraft on Vimeo.
If you want to use a usb drive to boot the machines, use Unetbootin to write the iso file onto the drive.
Burn the image files to a cd, then use them to boot up computers. On Windows machines you usually have to press one of the function keys during the bios, and on Intel-based iMac computers you need to hold down the 'c' key after the machine is turned on. The master machine must be booted first. Use login as 'tc'. When the nodes boot, they will ask for the master ip address and will drop into a terminal when they have successfully downloaded the master machine's public key. You don't need to do anything with the node computers after this point, but you can run the 'top' program to watch jobs on that node.
Once the machines are booted, run 'cbn reset' on the master to generate the machine file. The master generates the machine file by simply trying to ssh into each address that downloaded the master's public key. Before you can ssh yourself into the master machine (to transfer your files to/from there) you need to give it a password by running 'passwd'. To distribute your files to all the nodes, use 'cbn dist file1 file2 ...', and then execute your programs using 'cbn_run -np 4' or however many cpus your cluster has ('cbn_run' is a simple wrapper to 'mpi_run' that adds some extra flags so Open mpi doesn't complain about not having fancy pants libraries every time you run it).
Cluster by Night is generated by remastering a Tiny Core Linux release, which can be done on any *nix with the appropriate tools (cpio, tar, gzip, advdef, and mkisofs). Everything but the kernel is available in the “build your own” package listed above, or you can just get a look at the build scripts to see what's up.