There are two directories in the PyTrilinos package that provide python scripts for testing PyTrilinos, example and test. Generally speaking, unit tests go in the test directory and descriptive or demonstrative example scripts go in example.
Unit tests scripts shall begin with test, followed by the package name and an underscore, followed by a short description of the test, typically the class name being tested. For example:
testTeuchos_ParameterList testEpetra_MultiVector
are the base names of the unit tests for Teuchos.ParameterList and Epetra.MultiVector, respectively. In certain situations, the underscore and test description can be omitted.
Example scripts shall begin with ex, followed by the name of the primary package being demonstrated, followed by an underscore and a short description of the test. For example:
exEpetra_Comm exAztecOO_Operator
are the base names for example scripts demonstrating the Epetra.Comm class and an AztecOO solver that uses an Epetra.Operator, respectively. In certain situations, the underscore and test description can be omitted.
Running make in either the top, example or test build directories simply copies the test scripts from the source directory to the build directory. For historical reasons, the source files have the suffix .src and the build files have the suffix .py.
The CMakeLists.txt files in the test and example directories control which scripts get copied to the build directory. Each test script, or group of related test scripts, should be protected with IF() statements, depending on which PyTrilinos modules need to be present for the script to run.
To run all of the enabled tests, first make sure PyTrilinos and all tests and examples are up-to-date:
$ cd BUILD/packages/PyTrilinos $ make
Then you can use the CMake ctest program to run all of the tests:
$ ctest -W 10
The results of all of the tests can be found in the Testing directory, present in the directory from which ctest was run.
All test scripts shall use the optparse module to parse command line options and support the following options:
-t, --testharness test local build modules; prevent loading system-installed modules -v VERBOSITY, --verbosity=VERBOSITY set the verbosity level [default 2]
The -t option is to force use of the locally-built PyTrilinos, preventing the importing of any installed PyTrilinos modules. The verbosity option is used in all test scripts and optionally in any example scripts where it makes sense.
Tests scripts use the setpath module, local to both test and example directories, to set the python search path to find the local PyTrilinos modules. Locally built modules are imported directly:
import Teuchos import Epetra
while system-installed modules are imported like this:
from PyTrilinos import Teuchos from PyTrilinos import Epetra
the -t option is enforced in test scripts with the followning python code:
parser = OptionParser() parser.add_option("-t", "--testharness", action="store_true", dest="testharness", default=False, help="test local build modules; prevent loading system-installed modules") options,args = parser.parse_args() from testutil import fromPyTrilinosImport Epetra = fromPyTrilinosImport('Epetra', options.testharness)
Test scripts shall run in both serial or parallel. You may use either:
comm = Teuchos.DefaultComm.getComm()
or:
comm = Epetra.PyComm()
to obtain an appropriate communicator object for the test scripts. By convention, set the variable iAmRoot to either True or False depending on whether the communicator's rank is 0.
The test script shall output End Result: TEST PASSED if the test passes correctly. This helps the Trilinos test harness determine which tests pass and which do not, especially in parallel.
Unit tests are based on the unittest python library module. Test case classes inherit from unittest.TestCase. Individual tests are implemented as methods of these classes that begin with test. See the python documentation (http://www.python.org) for details.
Each unit test script can have one or more TestCase classes. In main(), each test case class should be added to a unittest.TestSuite object named suite.
Unit tests shall print, from processor 0, a header with a message such as "Testing Epetra.Object" with a line of asterisks above and below the message:
********************* Testing Epetra.Object *********************
The suite of tests should be run with:
verbosity = options.verbosity * int(iAmRoot) result = unittest.TextTestRunner(verbosity=verbosity).run(suite)
and the success should be determined and output via (for the case of an Epetra communicator):
errsPlusFails = comm.SumAll(len(result.errors) + len(result.failures)) if errsPlusFails == 0 and iAmRoot: print "End Result: TEST PASSED" sys.exit(errsPlusFails)
If there are no errors and no failures on any processors, then the test will pass.
Example scripts are more free-form and should be written for readability, to make for clear demonstrations of PyTrilinos usage. However, it is encouraged that example script output be consistent with unit test output whenever possible.