Blog
Development

Writing tests like a novelist

-

Last week my colleague Piet claimed: “You shouldn’t need several hours to understand what a method, class or package does”. Since unit tests are written in classes and methods, the same holds here. Welcome to the next episode of “reducing mental effort for software developers”.

In this post I will lay out how AssertJ can help to reduce the mental effort needed while reading and writing test code, and as a bonus how it reduces the effort needed for understanding results of failing tests. AssertJ is a library that provides fluent assertions for Java. Before I dive into the fluent part, let’s start with some examples of assertions. Suppose you want to check that a String is of a certain value. In JUnit this will be done in the following way:

assertEquals("expected", "result");

In natural language this statement can be described as: “assert that expected and result are equal”. The same check with AssertJ can be done with:

assertThat("result").isEqualTo("expected");

Comparing to JUnit, the two values are in a reversed order. With assertThat() you specify which value you want to check, followed by isEqualTo() you specify to which value it should comply. Now the statement is expressed in a way closely to that of natural language. If you would strip the punctuation marks and “de-CamelCase” it, you’ll get the sentence: “assert that result is equal to expected”. My English may not be perfect, but this statements sounds a lot more like a sane and natural sentence. Because the Strings of these two examples are unequal, these tests will fail with the message:

org.junit.ComparisonFailure:
Expected :expected
Actual   :result

Sometimes I come across unit tests where expected and result are swapped like this:

assertEquals("result", "expected");

This is correct, but can be confusing when you’ve broken some tests and reading the message:

org.junit.ComparisonFailure:
Expected :result
Actual   :expected

In this example it’s quite obvious that something is wrong in the test, but imagine that in more obscure situations you’ll need a lot more mental effort before you find out what’s wrong and why the test is failing. AssertJ does not offer bullet proof protection against these kind of programming errors, but it will reduce the chance. A bell should ring when you read or write:

assertThat("expected").isEqualTo("result");

These equals checks are simple examples to make a clear difference between plain JUnit and the fluent assertions of AssertJ. The real power of fluent kicks in when applying multiple assertions in one single statement. For example:

assertThat(alphabet)
       .isNotNull()
       .containsIgnoringCase("A")
       .startsWith("abc")
       .endsWith("xyz");

As we’ve seen before, this statement reads like natural language. In JUnit on the other hand the equivalent test will read like:

assertNotNull(alphabet);
assertTrue(alphabet.toUpperCase().contains("A"));
assertTrue(alphabet.startsWith("abc"));
assertTrue(alphabet.endsWith("xyz"));

Apart from needing four separate statements, we now discover that JUnit provides quite a limited API. Bluntly, JUnit can check that something is true/false or that something is null (or not). Using only JUnit we can’t say: “check that this String contains the character A”. We have to use the contains method of Java’s String class, and then check that its result is true. Let’s zoom in on the example of contains(). The JUnit the test:

assertTrue("abc".contains("A"));

will fail with the message:

java.lang.AssertionError
    at org.junit.Assert.fail(Assert.java:86)
    at org.junit.Assert.assertTrue(Assert.java:41)
    at org.junit.Assert.assertTrue(Assert.java:52)
    at assertj.Strings.contains_junit(StringsTest.java:34)
...

With Fluent Assertions the same test would be written as:

assertThat("abc").contains("A");

Because we exactly tell what we want to test (that “abc” contains the character A), AssertJ has enough information to tell us what went wrong. So this test fails with the message:

java.lang.AssertionError:
Expecting:
 <"abc">
to contain:
 <"A">

We’ve now seen how we can write better readable tests which give more information when a test fails. Until now I only gave examples with Strings, but AssertJ provides API’s for more data types. All examples can be found on AssertJ’s website, but let me highlight another commonly used data type.

Collections

Suppose we want to test this List of Strings:

List numberList = Arrays.asList("One", "Two");

In JUnit this will look like:

assertEquals(Arrays.asList("Two"), numberList);

And this fails with the message:

Expected :[Two]
Actual   :[One, Two]

Using AssertJ the same would look like:

assertThat(numberList).containsExactly("Two");

and this fails with the message:

Actual and expected should have same size but actual size was:
  <2>
while expected size was:
  <1>
Actual was:
  <["One", "Two"]>
Expected was:
  <["Two"]>

So AssertJ tells us that the size is incorrect. Nice, we do not have the scan all the elements to find out what the difference is ourselves. Another example where the size is equal, but the ordering is different. JUnit’s:

assertEquals(Arrays.asList("Two", "One"), numberList);

will fail with:

Expected :[Two, One]
Actual   :[One, Two]

While AssertJ’s:

assertThat(numberList).containsExactly("Two", "One");

will fail with:

Actual and expected have the same elements but not in the same order, at index 0 actual element was:
  <"One">
whereas expected element was:
  <"Two">

In these examples the lists only contained two elements, but when the list is larger, it will get hard to find out which element is missing, or to see the difference. A last example where the difference in Collections is a bit more obscure. Suppose we want to check if the following List of numbers correctly counts up:

List largeNumberList = Arrays.asList(1, 2, 2, 4, 5);

JUnit’s:

assertEquals(Arrays.asList(1, 2, 3, 4, 5), largeNumberList);

will fail with:

Expected :[1, 2, 3, 4, 5]
Actual   :[1, 2, 2, 4, 5]

Unless you become happy from playing a game of spot the difference this results in needless occupation of your mental capacity. And that while AssertJ’s:

assertThat(largeNumberList).containsExactly(1, 2, 3, 4, 5);

fails with:

Expecting:
  <[1, 2, 2, 4, 5]>
to contain exactly (and in same order):
  <[1, 2, 3, 4, 5]>
but could not find the following elements:
  <[3]>

In a glance we see what is wrong. Again, when Collections tends to be larger in size, these kind of failure messages are only getting more helpful.

Why not Hamcrest?

Well fair point. Hamcrest core has been included in JUnit since version 4.4 and tests using the hamcrest API look a lot more like AssertJ than that they look like plain JUnit. Also the failure messages are better than in Plain JUnit. But in my opinion Hamcrest does both these jobs not as well as AssertJ. Let’s compare the two.

Comparing Strings with Hamcrest:

assertThat("abc", containsString("A"));
fails with:
Expected: a string containing "A"
but: was "abc"

At least we see the expected (containing “A”) and actual ( “abc” ) here, so that’s better than JUnit. At this point Hamcrest still reads like natural language just like the Fluent Assertions. But let’s get back on the example with multiple assertions on the letters of the alphabet String.

With Fluent Assertions we saw:

assertThat("abc")
    .isNotNull()
    .startsWith("abc")
    .endsWith("xyz");

which fails with:

Expecting:
    <"abc">
to end with:
    <"xyz">

The equivalent in Hamcrest will look like: 

assertThat("abc", allOf(
    is(notNullValue()),
    startsWith("abc"),
    endsWith("xyz")));

and fails with:

Expected: (is not null and a string starting with "abc" and a string ending with "xyz")
    but: a string ending with "xyz" was "abc"

Decide for yourself which failure message requires less effort to understand what is tested and what went wrong. As we can see in the test itself, Hamcrest provides a prefix notation like API to perform multiple assertions. This requires the reader to create a mental model of a stack with the operators like allOf() and is() while understanding the different assertions. With the given example, this may sound exaggerated, but in more complex situations, this requires quite some mental effort.

As I said in the beginning, only Hamcrest-core is part of JUnit, which is quite limited. When you want to test collections, for example, you need to add hamcrest-all to your project. And when already adding an extra dependency to your project anyway, why not choose AssertJ. The last release of Hamcrest dates back to 2012, while AssertJ is more actively developed (May 2017) and supports Java 8 features.

The last reason why I think AssertJ is the best, the only, and nothing but the best is code completion is the additional advantage of its Fluent API so that we can simply use code completion to explore all the possibilities. Without the the need for memorizing the whole API or the need for cheat sheets.

Getting Started

The AssertJ website is filled with examples and instructions on how to include AssertJ in your project. For an extensive set of examples, see the assertj-examples tests project on GitHub. When you’re using Eclipse, use this tip to get code completion. You could do the same for Mockito. by the way.

While the examples in this post were in Java with the AssertJ library, the same ideas apply for other languages. See, for example, fluentassertions.com for .NET.

After reading this, I hope you’re even more devoted to creating code that is simple and direct. Or as Grady Booch, author of Object Oriented Analysis and Design with Applications, said, “clean code reads like well-written prose.”