CbN is a Linux live distribution that allows one to run standard desktop hardware as an MPI computing cluster without modification to the local hard disk or network topology (i.e. cluster nodes can receive their IP address from an existing dhcp server). The distribution is minimal—there is no support for NFS or a graphical desktop environment. The master node resides in 30 MB of ram, and the compute nodes even less. Compile an executable or write a Ruby or Lua script, send it to the master node, and distribute/run it on all the nodes automatically via SSH. The distribution is based on Tiny Core Linux
This project was initially inspired by the far more feature-rich Pelican HPC distribution, though no code is shared.
Cluster by Night Screencast from Dept. Dirigible Flightcraft on Vimeo
Burn the image files to a cd, then use them to boot up computers. On Windows machines you usually have to press one of the function keys during the BIOS and on Intel-based iMac computers you need to hold down the ‘c’ key after the machine is turned on. If you want to use a USB drive to boot the machines, use Unetbootin to write the ISO file onto the drive.
The master machine must be booted first. Login as ‘tc’. When the nodes boot, they will ask for the master ip address and will drop into a terminal when they have successfully downloaded the master machine’s public key. You don’t need to do anything with the node computers after this point, but you can run the ‘top’ program to watch jobs on that node.
Once the machines are booted, run ‘cbn reset’ on the master to generate the machine file. The master generates the machine file by simply trying to ssh into each address that downloaded the master’s public key. Before you can ssh yourself into the master machine (to transfer your files to/from there) you need to give it a password by running ‘passwd’. To distribute your files to all the nodes, use ‘cbn dist file1 file2 …’, and then execute your programs using ‘cbn_run -np 4’ or however many CPUs your cluster has (‘cbn_run’ is a simple wrapper to ‘mpi_run’ that adds some extra flags so OpenMPI doesn’t complain about not having fancy pants libraries every time you run it).