Subsections

3.3 PVM

PVM or parallel virtual machine is a freely available library for running processes on more than one processor on one or more machines. ROCS can take advantage of PVM to distribute jobs over multiple processors. To do this PVM must be installed on all the machines ROCS will be distributed over. The PVM source is freely available from

http://www.csm.ornl.gov/pvm/pvm_home.html

however many Linux distributions, and some Unix versions, include PVM by default. ROCS is built with the current PVM version 3.4.5, but should also work with PVM version 3.4.4. ROCS does not support PVM under Windows.

To use ROCS with PVM you must do one of the following.

Place a link or copy of the ROCS executable in $PVM_ROOT/bin/$PVM_ARCH
Define the enviroment variable PVM_PATH, which names the directory where the ROCS executable resides.

The enviroment variables PVM_ROOT and PVM_ARCH should be defined globablly as part of the PVM installation. PVM_PATH is generally a user-defined enviroment variable, and must defined for all shells (i.e., you can't just define it in the shell you're launching ROCS in).

NOTE : There is no specific slave executable. The executable distributed for this program serves as both a master and slave PVM program as well as a single processor version.

3.3.1 PVM hosts file

After PVM and ROCS are installed, each ROCS job run via PVM must be supplied a file (via -pvmconf) that describes the hosts and number of CPU's to use. Each line in this file includes the word ``host'' followed by the hostname of the slave followed by the number of CPU's to use on that slave.

To use 10 slaves on a linux cluster (where nodes are named ``linux1'', ``linux2'', etc.)

host linux1 2
host linux2 2
host linux3 2
host linux4 2
host linux5 2

To use 32 CPU's on a single-image system like an SGI Altix, a single line can be used:

host altix 32

3.3.2 PVM Scaling

PVM jobs involve a certain of network traffic, sending multi-conformer molecules from the master to the slaves and sending results from the slaves back to the master. The single ROCS slave jobs are relatively fast compared to the network I/O time, so for scaling beyond 32 CPUs, additional effort is required in setting up a job. For low numbers of CPUs or for divide-and-conquer style applications, using .oeb.gz as the dbase file format is fine.

As of version 2.0, Omega can write rotor-offset-compressed OEB files (via the -roc command-line flag) that greatly reduce the I/O on the master and allow ROCS to scale to larger number of CPUs.

An additional variable for PVM scaling is to modify the number of molecules sent to each slave using the -pvmpass command line option. In this version of ROCS, the default for -pvmpass has been changed to 5 based on some scalability tests on a couple of large clusters, but each site can make small changes here to find the best number for local conditions.