Are you a Mockist or a Classicist?

I recently re-read Martin Fowler’s essay ‘Mocks aren’t Stubs’ comparing the benefits and pitfalls of writing unit tests using either state verification or behavioural verification. He describes state verification as classical or regular unit testing, in that you run a test and check the state of your test object (as defined by the data a particular method call returns) using some kind of assert(…) method.

A Classic JUnit Test

@Test
public void testFlowStats1() {
 
    final int loopVal = 51;
 
    // Set the object up
    instance.reset();
 
    for (int i = 0; i < loopVal; i++) {
        instance.addDataPoint();
    }
     
    // Call the method under test
    TimerStatistics.TimerResults result = instance.calculateStats();
 
    // Verify the results 
    assertEquals(result.getAvgRecentIntervalTime(), result.getAvgTotalInterval(), 0);
    assertEquals(loopVal - 1, result.getDataPoints());
    assertEquals(loopVal - 1, result.getRecentCountSize());
}

Mockists, on the other hand, write unit tests that test the behaviour of code by describing a path through the code, calling the method under test and verifying that the code actually took the described path.

A Mockist JUnit Test

/**
 * This test uses Easymock and Unitils
 */
@Test
public void testManagerStartSequenceTaskCompleted() throws ExecutionException {

    // setup methods called by run()
    timerStats.addDataPoint();
    expect(executorService.submit(dummyTask)).andReturn(future);
    expect(future.isDone()).andReturn(true);
    expect(future.isCancelled()).andReturn(false);

    // Setup internal state
    CallReturnBean retBean = new CallReturnBean();
    retBean.endCall(true);
    expect(future.get()).andReturn(retBean);
    executionStats.addDataPoint(retBean);

    replay();

    instance.start();
    // call the instance and run the test
    instance.run();

    verify();
}

Reading the essay, I get the feeling that you’re supposed to choose: Mockist or Classicist, it's a bit like being either Roundhead or Cavilier, or choosing between Laural or Hardy; Morecambe or Wise; Lennon or McCartney.

Personally, I believe that these two approaches compliment each other; hence, I tend to be neither a Mockist nor Classicist, but more of a pragmatist. I like the testing isolation provided by mock objects, but also like to ensure that my classes work with their collaborators, which means using some classical state validation tests.

So, my current unit test strategy is to write a large number of mock based tests that test every route through the code, to ensure all scenarios work. These are highly isolated, fine grained tests that run very quickly and should be run most often.

When I’m confident that my class works, I also like to add a smaller number of classical or state verification tests that ensure my class works with its collaborators. This is doubly important if you’re using Spring to create a dependency injection network as it’s much quicker to load a Spring configuration on your local machine rather than to test it by deploying to your JEE server. Some would classify these coarser grained tests as mini-integration tests.

There are times, however, when mocks don’t provide the best solution and when classical tests come to in to their own. For example, in an enterprise application, at some point you’ll need a few tests that ensure your application can talk to the database and that it can read and write data appropriately.

Another case where mocks seem inappropriate is when your instance under test performs some kind of calculation, for example it may calculate some statistics. In this case you’ll need to check the result returned by your method, rather than the route taken through the code.

So, as I said, programming tests should come down to pragmatism, doing what’s simplest and easiest in order to get the code written and delivered.