Installation and Platform Notes¶
Licenses¶
To run BROOD you will need to obtain a license file for BROOD from OpenEye Scientific Software (business@eyesopen.com). The license file will be one of the 2 options:
- In a file pointed to by the environment variable OE_LICENSE.
- Named oe_license.txt and must be placed in the directory specified by the environment variable OE_DIR.
If you intend on running multi-processor BROOD via Open MPI, only the master machine needs access to the license file.
On Windows, the environment variables can be set under the system Control Panel.
General Installation¶
By default, all OpenEye applications are installed into a single distribution directory tree on the specified machine. The default location for this tree is platform specific and will be detailed below.
The root of the tree (i.e. the openeye directory) contains the following subdirectories:
| admin: | This directory is intended to contain any administrative scripts and tools associated with the installed applications. Currently, this directory is simply a placeholder on all platforms except for Microsoft Windows, where it contains the uninstaller executables. |
|---|---|
| arch: | This directory contains the collection of platform specific subdirectories. Each subdirectory contains the actual installed executables and support libraries for the associated platform. In the platform specific subdirectory, there will be a subdirectory for each application and within that will be another subdirectory for each version of that application. |
| bin: | This directory contains a startup script for each application that has been installed. This script determines at run-time what the current platform is and then calls the appropriate executable in the arch. This script enables the easy co-existence of multiple platforms and versions of any OpenEye application in the same distribution tree. |
| data: | This directory contains all of the associated data for the installed applications. There will be a subdirectory for each installed application and within that subdirectory there will be another subdirectory for each specific version of that application. |
| docs: | This directory contains all of the documentation associated with the installed applications. There will be a subdirectory for each installed application and within that subdirectory there will be another subdirectory for each specific version of that application. |
| examples: | This directory contains all of the examples associated with the installed applications. There will be a subdirectory for each installed application and within that subdirectory there will be another subdirectory for each specific version of that application. |
The startup script discussed in the section on the bin directory above will have the same name as the installed executable with which it is associated. When the script is called, it will attempt to determine the current platform and run the appropriate executable if installed. If an appropriate executable cannot be found, the script will report that information as well as a list of the currently installed platforms. The auto-detection can be overridden by setting one of two environment variables:
OE_ARCH can be used to specify a colon separated list of compatible distributions for the current platform such as:
redhat-RHEL5-x64:redhat-RHEL4-x64
Specification of this environment variable overrides the auto-detection process if it is present. If none of the compatible distributions listed are found, the script will fall back to the auto-detection process.
APPNAME_OE_ARCH can be used to specify a colon separated list of compatiable distributions for a specific application (as specified by changing the APPNAME text in the environment variable name) just like OE_ARCH as detailed above.
Specification of this environment variable overrides the OE_ARCH environment variable as well as the auto-detection process. If none of the compatible distributions listed are found, the script will fall back to the OE_ARCH list first and then to the auto-detection process.
Specifying this variable provides a simple way to customize the behavior for individual applications on non-standard platforms.
The startup script also supports a few commandline arguments including:
| -path | Specifying this argument will output the full path of the executable to be run. The executable will not be started if this argument is present. |
| -print_arch | Specifying this argument will output the details of the current platform as detected by the script as well as which platform-version of the executable is being run. The executable will be started if this argument is present. |
| -use_version | Specifying this argument followed by a specific version number allows the user to control which released version of the executable to run. |
Linux/Unix¶
Linux/Unix distributions are provided as a gzipped tarball of the distribution tree described above. Installation is performed by untarring the file in the desired location. Multiple distributions can be installed in the same location without any challenge.
To ensure that the installed applications can be called from the command line, be sure to add the full path of the openeye/bin subdirectory to the PATH environment variable. For instance, if the distribution was installed into /usr/local/openeye, the PATH environment variable should contain: /usr/local/openeye/bin.
Windows¶
Windows distributions are provided as a standard EXE installer. By default, all OpenEye applications will install into the C:\OpenEye directory.
An OpenEye group with an application specific subgroup will be added to the Start menu. The application specific subgroup will contain links to the documentation, the uninstaller, as well as to a Windows command shell which has the appropriate PATH settings already defined to allow the user to simply type the executable name at the prompt without concern for where the executable is actually installed.
For GUI applications, a link to the application will be created on the desktop as well as in the application specific subgroup of the Start menu.
Mac OS X¶
Mac OS X distributions are provided as a standard pkg installer delivered as a dmg disk image. By default, all OpenEye applications will install into the /Applications/OpenEye directory.
To ensure that the installed applications can be called from the command line in the Terminal, be sure to add /Applications/OpenEye/bin to the PATH environment variable.
For GUI applications, an application bundle which can be clicked on to start, will be present in the /Applications/OpenEye directory. This bundle cannot be moved independent of the OpenEye directory. For instance, the entire OpenEye directory can be moved as one piece, but moving the application bundle or the contents of any of the subdirectories in the OpenEye directory may cause the application to not start. However, the bundle can still be dragged into the Dock and run from there without any problem.
Open MPI¶
BROOD supports MPI-1, or Message Passing Interface 1, on all platforms except Microsoft Windows. MPI enables distributing BROOD runs over multiple machines, cores and processors. In addition, the CHOMP database preparation program is MPI enabled. The discussion below is focused on BROOD, but all of the MPI commands and concepts work analogously with the CHOMP program.
BROOD uses the Open MPI implementation of MPI, found at http://www.open-mpi.org. Every version of BROOD includes a full Open MPI install, so no additional software is needed.
There are two requirements to run BROOD under Open MPI.
- Every machine in the cluster must have disk-access to a BROOD installation. This access is used to load the program once at the beginning of execution and not for database or output i/o, so access speed is not critical.
- The path to the top level OpenEye bin directory must be in the PATH enviornment variable on all machines, and before any other locations that may contain MPI executables (orted, mpirun, etc).
- The instance of BROOD that initiates the execution (the master), must have access to a BROOD license. The rest of the BROOD instances (the slaves) do not need access to a license.
Using Open MPI¶
Running BROOD under Open MPI makes use of a wrapper script oempirun located in the OpenEye bin directory.
To run BROOD under Open MPI:
prompt> oempirun [Open MPI options] brood [BROOD Options]
There are two common uses for multiprocessor BROOD. The first is on a workstation with multiple cores, while the second is on a large cluster. We will present examples of both cases here. On most large clusters, users must access the computational resources through queue systems. We will not discuss that layer of complexity here, but focus on the use of MPI and BROOD with the expectation that users will incorporate this into their own particular system.
First, let us imagine you have a workstation with 8 cores and would like to use 7 of those for a BROOD job. The MPI flag for the number of processes to use is -np and it is followed by an integer.
prompt> oempirun -NP 7 brood [BROOD Options]
This command-line will execute a BROOD search on your current machine using 7 processors.
For the second example, let’s imagine you have access to five machines of a cluster (named c1 through c5) each with 4 processors. Further, lets assume that you set up your default login shell on each of these machines to have a PATH environment variabel including a bin directory of a BROOD installation. Finally, lets assume you’ve logged into c1 and have a shell in a directory on a local disk where you have scratch space. For efficiency, you would want the database you are searching to also be on a local disk of the machine where you start the job, but this is not absolutely necessary.
Begin by generating a text file that will include the MPI hosts you plan to use and the number of proccesses on each (for this example, we’ll call this file hosts). The file should contain a line for each machine with the name of the machine, a space, then slots=N, where N is the number of processors for your run. For this example, you would want the file to look like:
c1 slots=4
c2 slots=4
c3 slots=4
c4 slots=4
c5 slots=4
Now the following BROOD command-line will start a job with 1 master and 19 slaves.
prompt> oempirun -hostfile hosts brood [BROOD Options]
The master will be on c1, where the job is being started. All the i/o for the run will be from the master machine, and all results and logging information will be combined.
Alternatively, to run 3 BROOD processes on the hosts larry, curly and moe:
prompt> oempirun -np 3 -hosts larry,curly,moe brood -query molecule.oeb -db database
The above command line will run 3 BROOD processes on the hosts larry, curly and moe. The master machine, which needs a license, would be larry. It is important to note that the brood passed to oempirun does not take a path, but instead can be found in your PATH environment variable.
Troubleshooting¶
In some instances, we have found that cluster machines with multiple ethernet connections can get confused during the startup of MPI runs. In these instances, the slave machines may appear to be running at 100%, but little or no progress is reflected in the master’s log files. This problem does not appear to strike randomly, but instead happens consistently to some systems and not to others. Thus, you should watch the log file during the initiation of the run the first time you run BROOD on a cluster. If you encounter this problem, please refer to the -mca btl_tcp_if_include command on the faq link below.
http://www.open-mpi.org/faq/?category=tcp
In addition, please contact support@eyesopen.com.
Open MPI License¶
Most files in this release are marked with the copyrights of the
organizations who have edited them. The copyrights below generally
reflect members of the Open MPI core team who have contributed code to
this release. The copyrights for code used under license from other
parties are included in the corresponding files.
Copyright (c) 2004-2008 The Trustees of Indiana University and Indiana
University Research and Technology
Corporation. All rights reserved.
Copyright (c) 2004-2008 The University of Tennessee and The University
of Tennessee Research Foundation. All rights reserved.
Copyright (c) 2004-2008 High Performance Computing Center Stuttgart,
University of Stuttgart. All rights reserved.
Copyright (c) 2004-2007 The Regents of the University of California. All rights reserved.
Copyright (c) 2006-2008 Los Alamos National Security, LLC. All rights reserved.
Copyright (c) 2006-2008 Cisco Systems, Inc. All rights reserved.
Copyright (c) 2006-2008 Voltaire, Inc. All rights reserved.
Copyright (c) 2006-2008 Sandia National Laboratories. All rights reserved.
Copyright (c) 2006-2008 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Copyright (c) 2006-2008 The University of Houston. All rights reserved.
Copyright (c) 2006-2008 Myricom, Inc. All rights reserved.
Copyright (c) 2007-2008 UT-Battelle, LLC. All rights reserved.
Copyright (c) 2007-2008 IBM Corporation. All rights reserved.
Copyright (c) 1998-2005 Forschungszentrum Juelich, Juelich Supercomputing
Centre, Federal Republic of Germany
Copyright (c) 2005-2008 ZIH, TU Dresden, Federal Republic of Germany
Copyright (c) 2007 Evergrid, Inc. All rights reserved.
Copyright (c) 2008 Institut National de Recherche en
Informatique. All rights reserved.
Copyright (c) 2007 Lawrence Livermore National Security, LLC.
All rights reserved.
Copyright (c) 2007-2008 Mellanox Technologies. All rights reserved.
Copyright (c) 2006 QLogic Corporation. All rights reserved.
Additional copyrights may follow
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer listed
in this license in the documentation and/or other materials
provided with the distribution.
- Neither the name of the copyright holders nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
The copyright holders provide no reassurances that the source code
provided does not infringe any patent, copyright, or any other
intellectual property rights of third parties. The copyright holders
disclaim any liability to any recipient for claims brought against
recipient by any third party for infringement of that parties
intellectual property rights.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
