System requirements
Building the Zoltan library
Testing the Zoltan library
Reporting bugs in Zoltan
Incorporating Zoltan into Applications
Building applications that use Zoltan
Data types for global and local IDs
C++ interface
F90 interface
cd zoltan/BUILD_DIROptions to the configure command allow paths to third-party libraries such as ParMETIS, PT-Scotch and PaToH to be specified. Building with MPI compilers (e.g., mpicc) is the default for Autotools builds of Zoltan; many options allow specification of particular MPI paths and compilers.
../configure {options described below}
make everything
make install
Users desiring a Fortran90 interface to Zoltan must specify the "--enable-f90interface" option.
All options can be seen with the following command issued in the zoltan/BUILD_DIR directory:
../configure --help
The following script is an example of configuration and build commands using Autotools. It specifies that Zoltan should be built with both the ParMETIS and PT-Scotch interfaces. Paths to both ParMETIS and PT-Scotch are given. The prefix option states where Zoltan should be installed; in this example, Zoltan's include files will be installed in /homes/username/zoltan/BUILD_DIR/include, and the libraries will be installed in /homes/username/zoltan/BUILD_DIR/lib. This examples assumes the script is run from /homes/username/zoltan/BUILD_DIR.
#
../configure \
--prefix=/homes/username/zoltan/BUILD_DIR \
--with-gnumake \
--with-scotch \
--with-scotch-incdir="/Net/local/proj/zoltan/arch/all/src/Scotch5" \
--with-scotch-libdir="/Net/local/proj/zoltan/arch/linux64/lib/openmpi/Scotch5" \
--with-parmetis \
--with-parmetis-incdir="/Net/local/proj/zoltan/arch/all/src/ParMETIS3" \
--with-parmetis-libdir="/Net/local/proj/zoltan/arch/linux64/lib/openmpi/ParMETIS3"
make everything
make install
More examples are in the directory zoltan/SampleConfigurationScripts.
After the configuration is done in the build directory, object files and executables can be removed with make clean; the same configuration will be used for subsequent builds. Configuration information is removed with make distclean.
cd Trilinos/BUILD_DIRSerial builds are the default in Trilinos; for serial builds, Zoltan builds and links with the siMPI library provided by Pat Miller in the Zoltan distribution. More commonly, Zoltan users desire parallel builds with MPI libraries such as OpenMPI or MPICH. For such builds, users must specify CMake option
cmake \
-D Trilinos_ENABLE_ALL_PACKAGES:BOOL=OFF \
-D Trilinos_ENABLE_Zoltan:BOOL=ON \
{options described below} \
..
make
make install
-D TPL_ENABLE_MPI:BOOL=ONTrilinos also defaults to using a Fortran compiler, but Fortran is not required to build Zoltan; the option to disable this check is
Other options to the cmake command allow paths to third-party libraries such as ParMETIS, PT-Scotch and PaToH to be specified.
Users desiring a Fortran90 interface
to Zoltan must
specify the option
-D Zoltan_ENABLE_F90INTERFACE:BOOL=ON
All options can be seen with the following command issued in the Trilinos/BUILD_DIR directory:
rm CMakeCache.txt
cmake -LAH -D Trilinos_ENABLE_Zoltan:BOOL=ON ..
The following script is an example of configuration and build commands using CMake. It specifies that Zoltan should be built with both the ParMETIS and PT-Scotch interfaces. Paths to both ParMETIS and PT-Scotch are given. The prefix option states where Zoltan should be installed; in this example, Zoltan's include files will be installed in /homes/username/Trilinos/BUILD_DIR/include, and the libraries will be installed in /homes/username/Trilinos/BUILD_DIR/lib. This examples assumes the script is run from /homes/username/Trilinos/BUILD_DIR.
#
cmake \
-D CMAKE_INSTALL_PREFIX:FILEPATH="/home/username/Trilinos/BUILD_DIR" \
-D TPL_ENABLE_MPI:BOOL=ON \
-D CMAKE_C_FLAGS:STRING="-m64 -g" \
-D CMAKE_CXX_FLAGS:STRING="-m64 -g" \
-D CMAKE_Fortran_FLAGS:STRING="-m64 -g" \
-D Trilinos_ENABLE_ALL_PACKAGES:BOOL=OFF \
-D Trilinos_ENABLE_Zoltan:BOOL=ON \
-D Zoltan_ENABLE_EXAMPLES:BOOL=ON \
-D Zoltan_ENABLE_TESTS:BOOL=ON \
-D Zoltan_ENABLE_ParMETIS:BOOL=ON \
-D ParMETIS_INCLUDE_DIRS:FILEPATH="/home/username/code/ParMETIS3_1" \
-D ParMETIS_LIBRARY_DIRS:FILEPATH="/home/username/code/ParMETIS3_1" \
-D Zoltan_ENABLE_Scotch:BOOL=ON \
-D Scotch_INCLUDE_DIRS:FILEPATH="/home/username/code/scotch_5.1/include" \
-D Scotch_LIBRARY_DIRS:FILEPATH="/home/username/code/scotch_5.1/lib" \
..
make
make install
More examples are in the directory zoltan/SampleCmakeScripts.
More details of CMake use in Trilinos are
in
Trilinos/cmake/TrilinosCMakeQuickstart.txt.
The "right" answer for these tests depends on the number of processes with which you run the tests. In general, if they compile successfully, run quickly (in seconds), and produce reasonable looking output, then Zoltan is built successfully.
The C++ interface to Zoltan is implemented in header files which define classes that wrap the Zoltan C library. The file include/zoltan_cpp.h defines the Zoltan class which encapsulates a load balancing data structure and the Zoltan load balancing functions which operate upon it. Include this header file instead in your C++ application. Note that C++ applications should call the C function Zoltan_Initialize before creating a Zoltan object.
Fortran applications must USE module zoltan and specify the Zoltan installation's include directory as a directory to be searched for module information files.
The C, C++ or Fortran application should then be linked with the Zoltan library (built with Fortran support in the Fortran case) by including
-lzoltanin the linking command for the application. Communication within Zoltan is performed through MPI, so appropriate MPI libraries must be linked with the application. Third-party libraries, such as ParMETIS, PT-Scotch and PaToH, must be also be linked with the application if they were included in compilation of the Zoltan library.
The installed files include/Makefile.export.zoltan* contain macros that can specify Zoltan paths and libraries in an application's Makefiles. Using these files, applications can be assured they are using the same build options that were used when Zoltan was built.
The following type definitions are defined in include/zoltan_types.h; they can be used by an application for memory allocation, MPI communication, and as arguments to load-balancing interface functions and application-registered query functions.
typedef unsigned int ZOLTAN_ID_TYPE;In the Fortran interface, IDs are passed as arrays of integers since unsigned integers are not supported in Fortran. See the description of the Fortran interface for more details.
typedef ZOLTAN_ID_TYPE *ZOLTAN_ID_PTR;
#define ZOLTAN_ID_MPI_TYPE MPI_UNSIGNED
The local IDs passed to Zoltan are not used by the library; they are provided for the convenience of the application and can contain any information desired by the application. For instance, local array indices for objects may be passed as local IDs, enabling direct access to object data in the query function routines. See the application-registered query functions for more details. The source code distribution contains an example application zdrive in which global IDs are integers and local IDs are local array indices. One may choose not to use local ids at all, in which case NUM_LID_ENTRIES may be set to zero.
Some Zoltan routines (e.g., Zoltan_LB_Partition and Zoltan_Invert_Lists) allocate arrays of type ZOLTAN_ID_PTR and return them to the application. Others (e.g., Zoltan_Order and Zoltan_DD_Find) require the application to allocate memory for IDs. Memory for IDs can be allocated as follows:
ZOLTAN_ID_PTR gids;The system call malloc may be used instead of ZOLTAN_MALLOC.
int num_gids, int num_gid_entries;
gids = (ZOLTAN_ID_PTR) ZOLTAN_MALLOC(num_gids * num_gid_entries * sizeof(ZOLTAN_ID_TYPE);