Writing tests like a novelist

Java

Last week my colleague Piet claimed: “You shouldn’t need several hours to understand what a method, class or package does”. Since unit tests are written in classes and methods, the same holds here. Welcome to the next episode of “reducing mental effort for software developers”.

In this post I will lay out how AssertJ can help to reduce the mental effort needed while reading and writing test code, and as a bonus how it reduces the effort needed for understanding results of failing tests. AssertJ is a library that provides fluent assertions for Java. Before I dive into the fluent part, let’s start with some examples of assertions.

Suppose you want to check that a String is of a certain value. In JUnit this will be done in the following way:

Clean code reads like well-written prose

java
assertEquals("expected", "result");

In natural language this statement can be described as: “assert that expected and result are equal”. The same check with AssertJ can be done with:

java
assertThat("result").isEqualTo("expected");

Comparing to JUnit, the two values are in a reversed order. With assertThat() you specify which value you want to check, followed by isEqualTo() you specify to which value it should comply. Now the statement is expressed in a way closely to that of natural language. If you would strip the punctuation marks and “de-CamelCase” it, you’ll get the sentence: “assert that result is equal to expected”. My English may not be perfect, but this statements sounds a lot more like a sane and natural sentence. Because the Strings of these two examples are unequal, these tests will fail with the message:

java
org.junit.ComparisonFailure:
Expected :expected
Actual   :result

Sometimes I come across unit tests where expected and result are swapped like this:

java
assertEquals("result", "expected");

This is correct, but can be confusing when you’ve broken some tests and reading the message:

java
org.junit.ComparisonFailure:
Expected :result
Actual   :expected

In this example it’s quite obvious that something is wrong in the test, but imagine that in more obscure situations you’ll need a lot more mental effort before you find out what’s wrong and why the test is failing. AssertJ does not offer bullet proof protection against these kind of programming errors, but it will reduce the chance. A bell should ring when you read or write:

java
assertThat("expected").isEqualTo("result");

We don’t want to know if our expectation is correct! We want to know if the result is correct, i.e. that it meets our expectation.

These equals checks are simple examples to make a clear difference between plain JUnit and the fluent assertions of AssertJ. The real power of fluent kicks in when applying multiple assertions in one single statement. For example:

java
assertThat(alphabet)
       .isNotNull()
       .containsIgnoringCase("A")
       .startsWith("abc")
       .endsWith("xyz");

As we’ve seen before, this statement reads like natural language. In JUnit on the other hand the equivalent test will read like:

java
assertNotNull(alphabet);
assertTrue(alphabet.toUpperCase().contains("A"));
assertTrue(alphabet.startsWith("abc"));
assertTrue(alphabet.endsWith("xyz"));

Apart from needing four separate statements, we now discover that JUnit provides quite a limited API. Bluntly, JUnit can check that something is true/false or that something is null (or not). Using only JUnit we can’t say: “check that this String contains the character A”. We have to use the contains method of Java’s String class, and then check that its result is true. Let’s zoom in on the example of contains(). The JUnit the test:

java
assertTrue("abc".contains("A"));

will fail with the message:

java
java.lang.AssertionError
    at org.junit.Assert.fail(Assert.java:86)
    at org.junit.Assert.assertTrue(Assert.java:41)
    at org.junit.Assert.assertTrue(Assert.java:52)
    at assertj.Strings.contains_junit(StringsTest.java:34)
...

This does not give away any information about what is wrong. Something went wrong with contains, but what String was tested? And what did we expect it to contain? When this happens while running the test in your IDE you hopefully can click somewhere so that it jumps to the line where it failed (line 34 in StringsTest.java) so you can find the error by looking at the assertion statement. But when reading the test results report from a Continuous Integration server on the other hand you have no context...

With Fluent Assertions the same test would be written as:

java
assertThat("abc").contains("A");

Because we exactly tell what we want to test (that “abc” contains the character A), AssertJ has enough information to tell us what went wrong. So this test fails with the message:

java
java.lang.AssertionError:
Expecting:
 <"abc">
to contain:
 <"A">

Both in your IDE as on the CI server this will save a lot of time and mental effort because you see what’s wrong in a glance.

We’ve now seen how we can write better readable tests which give more information when a test fails. Until now I only gave examples with Strings, but AssertJ provides API’s for more data types. All examples can be found on AssertJ’s website, but let me highlight another commonly used data type.

Collections

Suppose we want to test this List of Strings:

java
List numberList = Arrays.asList("One", "Two");

In JUnit this will look like:

java
assertEquals(Arrays.asList("Two"), numberList);

And this fails with the message:

java
Expected :[Two]
Actual   :[One, Two]

Using AssertJ the same would look like:

java
assertThat(numberList).containsExactly("Two");

and this fails with the message:

java
Actual and expected should have same size but actual size was:
  <2>
while expected size was:
  <1>
Actual was:
  <["One", "Two"]>
Expected was:
  <["Two"]>

So AssertJ tells us that the size is incorrect. Nice, we do not have the scan all the elements to find out what the difference is ourselves. Another example where the size is equal, but the ordering is different. JUnit’s:

java
assertEquals(Arrays.asList("Two", "One"), numberList);

will fail with:

java
Expected :[Two, One]
Actual   :[One, Two]

While AssertJ’s:

java
assertThat(numberList).containsExactly("Two", "One");

will fail with:

java
Actual and expected have the same elements but not in the same order, at index 0 actual element was:
  <"One">
whereas expected element was:
  <"Two">

In these examples the lists only contained two elements, but when the list is larger, it will get hard to find out which element is missing, or to see the difference. A last example where the difference in Collections is a bit more obscure. Suppose we want to check if the following List of numbers correctly counts up:

java
List largeNumberList = Arrays.asList(1, 2, 2, 4, 5);

JUnit's:

java
assertEquals(Arrays.asList(1, 2, 3, 4, 5), largeNumberList);

will fail with:

java
Expected :[1, 2, 3, 4, 5]
Actual   :[1, 2, 2, 4, 5]

Unless you become happy from playing a game of spot the difference this results in needless occupation of your mental capacity. And that while AssertJ's:

java
assertThat(largeNumberList).containsExactly(1, 2, 3, 4, 5);

fails with:

java
Expecting:
  <[1, 2, 2, 4, 5]>
to contain exactly (and in same order):
  <[1, 2, 3, 4, 5]>
but could not find the following elements:
  <[3]>

In a glance we see what is wrong. Again, when Collections tends to be larger in size, these kind of failure messages are only getting more helpful.

Why not Hamcrest?

Well fair point. Hamcrest core has been included in JUnit since version 4.4 and tests using the hamcrest API look a lot more like AssertJ than that they look like plain JUnit. Also the failure messages are better than in Plain JUnit. But in my opinion Hamcrest does both these jobs not as well as AssertJ. Let’s compare the two.

Comparing Strings with Hamcrest:

java
assertThat("abc", containsString("A"));
fails with:
Expected: a string containing "A"
     but: was "abc"

At least we see the expected (containing “A”) and actual ( “abc” ) here, so that’s better than JUnit. At this point Hamcrest still reads like natural language just like the Fluent Assertions. But let’s get back on the example with multiple assertions on the letters of the alphabet String. With Fluent Assertions we saw:

java
assertThat("abc")
       .isNotNull()
       .startsWith("abc")
       .endsWith("xyz");

which fails with:

java
Expecting:
 <"abc">
to end with:
 <"xyz">

The equivalent in Hamcrest will look like:

java
assertThat("abc", allOf(
       is(notNullValue()),
       startsWith("abc"),
       endsWith("xyz")));

and fails with:

java
Expected: (is not null and a string starting with "abc" and a string ending with "xyz")
     but: a string ending with "xyz" was "abc"

Decide for yourself which failure message requires less effort to understand what is tested and what went wrong. As we can see in the test itself Hamcrest provides a prefix notation like API to perform multiple assertions. This requires the reader to create a mental model of a stack with the operators like allOf() and is() while understanding the different assertions. With the given example this may sound exaggerated, but in more complex situations this requires quite some mental effort.

As I said in the beginning only hamcrest-core is part of JUnit, which is quite limited. When you want to test collections for example you need to add hamcrest-all to your project. And when  already adding an extra dependency to your project anyway, why not choose assertj. Last release of Hamcrest dates back to 2012, while AssertJ is more actively developed (may 2017) and supports Java8 features.

Last reason why I think AssertJ is the best, the only and nothing but the best is code completion. Additional advantage of its Fluent API is that we can simply use code completion to explore all the possibilities. Without the the need for memorizing the whole API or the need for cheat sheets.

Getting Started

The website of AssertJ is full of examples and instructions on how to include AssertJ in your project. For an extensive set of examples see the assertj-examples tests project on Github.
When you’re using Eclipse, see this tip to get code completion. You could do the same for Mockito by the way 😉

While the examples in this post were in Java with the AssertJ library, the same ideas apply for other languages. See for example fluentassertions.com for .NET.

After reading this, I hope you’re even more devoted to create code that is simple and direct. Or as Grady Booch, author of Object Oriented Analysis and Design with Applications, said:

Java