Introduction to unit testing

Writing test cases are like levels in a video-game. There are easy test cases, and very hard ones. Only expert gamers kill the strongest monsters of the last level. Only expert programmers write the hardest tests using advanced techniques such as dependency injection and mocks.

In this document, the examples are given in Java and uses the JUnit test framework.

Level 0 Test - Happy Path Test

The level 0 tests specify the nominal behavior under standard conditions (also called the “happy path”).

A level 0 test contains three components:

the contract that is being tested
the setup of the test, initializing and executing the program under test
at least one assertion which compares the actual behavior against the expected behavior.

@Test void test() { // contract: the getter enables to retrieve the value passed to the setter. x = new Author() x.setName(“Toto”) assertEquals(“Toto”, x.getName()) }

Note that:

a test contains at least one assertion
a test specifies a contract, the contract must be written in natural language (some encode the contract in the test name but I find this not readable)
a contract may involve multiple methods/functions
the order of the assertion’s arguments is expected, then actual value. It is mandatory in order to have a clear error message when the assertion fails (‘expected: foo, but got bar’).

In a level 0 test:

the expected value, also called the oracle, is hard-coded
there are only local variables, object creations, method calls, one assertion
there is no if/then/else, try catch, loops, etc (test code is not normal code)
there is no setup/teardown code,

Antipatterns:

Using assertTrue(...==...): solution assertEquals(...,...).
x = new User(); assertNotNull(c). This does not test your code.
assertTrue(true)

Examples of contracts:

The function “fact” computes the factorial of an integer
Adding an element in a list enables one to find it afterwards
Adding an element in a list puts it at the end of the list

Level 1a Test: error cases

The level 1b tests specify the error cases. This can be done using exceptions or using return codes. Exceptions are better.

Only local variables, object creations, method calls, one assertion

  @Test(expected=IllegalArgumentException.class)
  void testNoNullName() throws Exception {
    x = new Author()
    x.setName(null)
  }

Note: the assertion here is not an assert* call but the expected value of the test annotation.

Forbidden constructs: setup/teardown code, if/then/else, try catch, loops, etc.

Level 1b test: your own assertions.

In some cases you need to define your own assertions methods. For instance, in a math library an assertMatrixEquals() that checks for matrix equality.

Level 2 test: using additional assertions libraries

There exists many different libraries to simplify assertions, eg. Harmcrest, and AssertJ. Example of a very readable assertion:

  assertThat(x.getName(),is("Toto"))

Level 3a test: mocks without libraries

Mocking can be used with and without libraries. To correctly use a library, one must first understand the basics of mocking. See Introduction to mocking for unit tests.

Only local variables, object creations, method calls, mock class declaration, one assertion

  @Test
  void testWebApp() {
    WebApp app = new WebApp();
  
    IDatabase mockDb = new DBMock();
    app.setDataBase(mockDb);
  
    // the user will be stored in the mock database
    app.createUser("Martin Monperrus", "University of Lille");
  
    assertEquals("University of Lille", app.getAffiliation("Martin Monperrus"));
  }

  // a mock key-value database 
  // instead of persisting to disk, all values  are stored in memory
  // here, the mock object is implemented as an anonymous class
  class DBMock implements IDatabase {
      private Map<String,String> data = new HashMap<>();
    
      @Override
      public void insert(String key, String value) {
        this.map.put(key, value);
      }
    
      @Override
      public String get(String key) {
        return this.map.get(key);
      }
  }

Level 3b test: mocks with libraries

There exists many mocking frameworks for all mainstream programming languages. For example, in Java there are JMock of Mockito. See the corresponding documentation.

Level 4a test: Plain parametrized tests

When one wants to test the contracts of an interface or a library, one needs a generic test, also called a parametrized test.

For instance, to test all contracts of a Java List, one may write:

  void testListContracts(List l) {
     ...
  }

  @Test
  void testArrayList() {
    testListContracts(new ArrayList());
  }

Level 4c test: Parametrized tests with Junit

Some testing libraries, incl. Junit. supports parametrized tests. Example:

  @RunWith(Parameterized.class)
  public class ParameterizedTest {

      @Parameters
      public static Collection<Object[]> data() {
          return Arrays.asList(new Object[][] { { new ArrayList<String>() },
                  { new Vector<String>() }, { new TreeSet<String>() } });
      }

      private Collection<String> col;

      public ParameterizedTest(Collection<String> col) {
          this.col = col;
      }

      @Test
      public void testParameter() {
          assertEquals(0, col.size());
          assertTrue(col.isEmpty());
          col.add("a string");
          assertEquals(1, col.size());
          assertFalse(col.isEmpty());
          col.clear();
          assertEquals(0, col.size());
          assertTrue(col.isEmpty());
      }
  }

Notes

What about setup (@Before) and tearDown (@After) code? Personally, I don’t like them, because they hinder the readability and maintainability of test code, because one has to scroll up and down to understand the test.

Acknowledgements

Daniel Le Berre