Installation
In order to run the testsuite, you need three things.
1. Test Suite Runner
Currently, there is no official way to run the SBML testsuite. However, we have a version of the unofficial java client. This works nicely thanks to the modifications by Frank Bergman. Please download the
No such attachment on this page and put in a place of your choice. Before proceeding start it with:
java -jar SBMLTestSuite.jar
This will create a configuration directory
.sbmltestrunner located in your home directory. Note, under Windows your home directory is
C:\Documents and Settings\<your account> which cannot be changed with the environment variable
$HOME. The testsuite runner includes a set of test cases which can be found in:
$HOME/.sbmltestrunner/2008-08-24/cases
or
$HOME/.sbmltestrunner/null/cases
2. The Test Suite
An official set of test cases is available via SVN at the
SBML SVN Server. To replace the test cases included in Test Suite Runner follow these instructions:
- change to:
$HOME/.sbmltestrunner/2008-08-24/cases or $HOME/.sbmltestrunner/null/cases
- remove:
semantic
- execute:
svn co https://sbml.svn.sourceforge.net/svnroot/sbml/trunk/test-suite/cases/semantic
This allows you to update the test cases through svn when fixes or new tests become available.
3. The COPASI Wrapper
The source code for the wrapper is in a subdirectory called sbml-testsuite of the COPASI sources. If you have configured COPASI, you just enter that directory and call make.
Running the tests
In order to run the tests, you start a java virtual machine with the jar file
java -jar SBMLTestSuite.jar. The testrunner GUI will open and in the "File" menu you can find an item called "New Test Run". Calling that will present you with a wizard that lets you select which version of SBML you want to test (L2V3 is what I use) and which features you don't want to test. I usually exclude "algebraic rules" and "fast reactions". (There is more than one page to disable features!). Currently there is no way to disable "delay functions" in the wizard. So if there are any tests with delay functions, we will fail those, but I don't think that there currently are any.
In the last page of the wizard, you have to specify the full path to the test wrapper you compiled. As arguments to the wrapper you give
%d %n %o 2 3 where the last two number denote the level and version of SBML you chose to test. The other arguments are variables that specify the test directory, the test number and the output directory.
Once you have finished the wizard, you get to a widget with a little green run button and if you click that, the program will run all the tests you specified. To have a nice view of the results, you can select the "Map View" even while the tests are running.
Interpreting the results is rather obvious. Green rectangle means we passed the test, red rectangle means we failed the test and blue rectangle usually means that there either was no output from the wrapper (e.g. if it crashed) or that there was not enough output (e.g. the integrator stopped before the end). Also if there are NaN values in the file, it gets a blue rectangle.
On Windows I (Stefan) use:
<path to>\sbml-testsuite.exe %d %n %o 2 3
Results
We pass 840 of 840 test if we skip tests with "algebraic rules" and "fast reactions".
