Blog

Testing with BEASTShell

Leave a comment

16 June 2014 by Remco Bouckaert

One motivation for BEASTShell is to ease the burden of writing tests for BEASTObjects. Writing tests is never much fun, but a necessary requirement to gaurantee quality of software. Writing BEASTObject tests is more cumbersome than usual because there is a division of instantiating inputs and initialising the object, which is usually done in a constructor, but in BEASTObjects requires a call to initAndValidate. Now we have the BEASTShell and can construct BEASTObjects a lot easier, it should be able to write tests with a lot less code and effort.

Constructing BEASTObjects

One way to use BEASTShell is by having a BEASTShell interpreter and just use it to construct BEASTObjects with fragments of script. For example, a unit test for the Uniform distribution may look something like this

First, create an Interpreter

final bsh.Interpreter interpreter = new bsh.Interpreter();

Now, you can use the interpreter to create BEASTObjects, e.g.

// create the BEASTObject
Uniform d = (Uniform) interpreter.eval("new beast.math.distributions.Uniform(lower=1.0, upper=2.0)");

// use the BEASTObject as in any other test
assertEquals(1.5, d.getMean(), 1e-10);

You can use the assertEquals command from BEASTShell in your test as well.
Instead of the above, we can run the same test with

// first tell the interpreter to import classes
interpreter.eval("import beast.math.distributions.*");

// create beast.math.distributions.Uniform object and test
interpreter.eval("d = new Uniform(lower=1.0, upper=2.0);" +
		"assertEquals(1.5, d.getMean(), 1e-10);");

Tests in script files

If your tests become a bit more involved than just a line or two, you probably do not want to wrap the script in a Java string, where you need to escape all double quotes, and add quotes around invidual lines.

Just save the following script in a bsh file, say in file src/test/beast/shell/uniform.bsh

import beast.math.distributions.*
d = new Uniform(lower=1.0, upper=2.0);
assertEquals(1.5, d.getMean(), 1e-10);

You can create a JUnit test as per usual, and create an bsh.Interpreter to run the script, e.g, using

public class UniformTest extends TestCase {

	@Test
	public void testUniform() throws FileNotFoundException, EvalError {
		Interpreter interpreter = new Interpreter();
		File bshfile = new File("src/test/beast/shell/uniform.bsh");
		interpreter.eval(new FileReader(bshfile));
	}	
}

However, the test.beast.shell.ShellTest class provides a test environment that makes things even easier. Just create a test-class that derives from test.beast.shell.ShellTest and create a method that returns a TestSuite, like so

public class UniformTest extends ShellTest {
	
	public static junit.framework.Test suite() throws Exception {
		return addTests("src/test/beast/shell/uniform.bsh");
	}
}

The ShellTest will load TestHarness.bsh first and calls complet() defined in TestHarness.bsh — provided it gets to the end of the script. If something fails, instead of calling asserts, you can set super.test_failed = true; and ShellTest will indicate that the test did not pass.

Developing test scripts

I found that the easiest and fastest way to set up test scripts is to run tests interactively in BEASTStudio. The history records every command, and it is easy to copy and paste relevant bits of code into a script after playing with things in the console for a while.

The edit() command can help creating scripts, for example for creating a sequence interactively, use

bsh % import beast.evolution.alignment.*;
bsh % seq = new Sequence();
bsh % edit(seq);

A dialog pops up where you can set values of the sequence object.

After selecting OK, the seq variable gets assigned the values entered, but also the value/name pair sequence is printed, so you can copy and paste that into any script.

Leave a Reply