The software is currently delivered as an eclipse plugin that installs from your Help | Install New Software... menu. Once installed, it integrates into your build process, all you need to do is to enable SureAssert on your project and ensure that Build Automatically is enabled on the eclipse Project menu. It also comes with a couple of sample projects to get you started.
The sample projects cover everything you need to get you started, so in this blog I’m not going into too much detail, all I want to do is to demonstrate a few basic key concepts to show why I think that declarative tests are a good idea.
As usual I’ve got a very contrived scenario: a simple Calculator class with two methods: add(...) and multiply. I’m assuming that at this point everything’s installed correctly , Build Automatically is selected and that the project is being looked after by the SureAssert plugin.
public final class Calculator {
@UseCase(args = { "1", "2" }, expect = "3")
public int add(int x, int y) {
return x + y;
}
@UseCase(name = "multiplyTest", args = { "4", "2" }, expect = "8")
public int multiply(int x, int y) {
return x * y + 1;
}
}Looking at the code above, you notice a couple of things. Firstly, there are a couple of new annotations applied to the methods, and secondly, that there is an error in the multiply(...) method; the programmer, for some reason, added one to the result.
Taking the @UseCase annotation applied to the add(..) method: this defines two arguments, “1” and “2”, using the args attribute, applies them to the method and expects “3” as a result; i.e. 1+2=3.
Taking the @UseCase annotation applied to the multiply(..) method: this defines two arguments, “4” and “2”, using the args attribute, applies them to the method and expects “8” as a result; i.e. 4 x 2 = 8.
So what you may say. The next thing to do it to take a look at eclipse to see what the SureAssert eclipse plug in gives you:
Th results of SureAssert’s testing are shown in the screen shot above and you’ll notice the tick icon
by the add(...) method indicating a pass and the purple cross icon 
indicating that the multiply(...) method has failed. The unit test failure is also indicated by the usual red error cross on your class name in the package explorer: 
The above example is very simple, covering only the briefest introduction of the functionality offered by SureAssert declarative unit testing. It also provides support for stubbs of various types to isolate your code from its dependencies. If you want to know more take a look at the SureAssert website.
28th Aug 2011 Update. I've just checked the SureAssert website and version 1.2 is now available for download. This seems to be a bug fix release, although you can now replace the @UseCase annotation with the @Exemplar, which to me seems better as it both conveys the right meaning and also isn’t a loaded term.
I’ve said before that annotations were a solution looking for a problem and with declarative unit testing they’ve found yet another problem to solve.
11 comments:
Looks really good for single-method unit testing... now if only you could get something similar for integration testing.
I'd also be interested in how this style could be ported to other languages... trying to think how I could do something similar for ruby right now :)
For Python there is an Doctests - tests that are embedded in functions' docstring.
As far as I know, Ruby also has something similar.
That is a terrible abuse of annotations. Its bad enough to mix your configuration with your code. Now we're going to mix in our tests with our source?
looks very promising, yes unit testing is a pain, thank for highlighting it.
I really don't think that SureAssert is an abuse of annotations. The annotations merely tell SureAssert how to test your code. The SureAssert eclipse plugin generates and runs the tests.
Although I have do not vested interest in the product I do like the idea that tests are not just an afterthought added on after you've written your code - they should be an integral part of development and promoted to the level of compiler errors. I wouldn't go so far as wanting to mix test code (i.e. JUnit code) with the application code, so specifying tests as simple annotations that allow automatic test generation seems a good idea to me.
Python has similar concept, called doctest, but instead of annotations you have to use method level documentation. The problem with this approach is that this is fairly limited, basically you can test only pure functions, and java is not a functional language, thus interaction testing (mocking) is often needed. And mocking also has a lot of advantages, it can drive the design towards the good direction.
Adding test-related annotations to production code doesn't seem like a cool idea. It does look nice in clean for the simple examples though.
Python has a similar tool in doctest.
I wonder though how you are going to inject mock dependencies using this style without getting dirty.
Will you implement an interface within the annotations?
Interesting approach, where the test is used to also document tightly to the actual code
But I see problems:
The above prod-code is trivial. It is a static method and from itself well testable. I wonder how this works with with more complicated domain-objects or handler objects (based on interfaces), and with higher cyclomatic complexity (if/else logic). I remember recently I had 200 LOC (compact + non-duplicated) text-cases/test-code which tested 20 LOC prod-code. I really doubt that SureAssert scales...
But still thanks for sharing (I like new approaches to testing).
Thanks for the comments on scaling and mock/stub objects. Yes, this is a VERY simple example, but the SureAssert technique does provide for Test Doubles, Method Stubs and Source Stubs. For more information on this take a look at: http://www.sureassert.com/uc/features/declarative-stubbing/.
Disclosure – I wrote Sureassert, thanks to Captain Debug for the article!
manuel aldana – thanks very much for your comments. Of course a basic example is given for simplicity and brevity; there is way more powerful stuff you can do with Exemplars, in particular using results from one into the instance or input of others. A nice way to set your object under test is to use the result of an Exemplar defined on a constructor or factory method, or you may want to use a named Spring bean.
Three main points I want to make: firstly Exemplars are for unit tests, not integration or functional tests. These typically run more complex scenarios. Secondly, Exemplars don’t fit every unit test requirement (I make no claim they kill JUnit!). But they do promote the idea of formalizing standard unit tests into a typical template, namely:
a. Create or otherwise retrieve an object (the “object under test”),
b. Execute a method on the object under test with typical inputs, and
c. Assert one or more expected results on the method’s returned value, thrown exception, the object under test’s state, and/or the state of some other affected object.
Typically you might also:
d. Setup stubs or mocks to force external dependencies to behave in a manner determined by the unit test, and
e. Assert the behaviour of the method under test, e.g. what methods have been invoked by the method under test and in what way.
These are the things you define in your Exemplar (or UseCase), which has the potential wipe out a large percentage of your coded tests. The rest can still be managed by the Sureassert engine.
I would say, if you have unit tests that are on average 10 times more logically complex than your production code, you may have a problem with the complexity of your unit test approach (although I can’t purport to know your specific circumstances of course!) If you have a test with a branch, you may well have 2 tests in one test method which would be represented by 2 separate Exemplars.
Thirdly, with an Exemplar, if necessary you’re quite welcome to call out to a method in your test source to make complex assertions on the Exemplar result or the test instance, via a simple invocation expression. The idea is to simplify typical tests such that this isn’t usually necessary however. You may consider Exemplars a good approach to supporting smaller, modular and more focussed and specified units in your production code.
Apologies that the site is lacking a little in documentation thus far. I’m working on a quick guide that will introduce the concepts, features and techniques.
To the contributor who didn’t like annotations in code – might not win you over, but there is the option of annotating your interfaces instead – Exemplars get inherited (potentially from multiple interfaces). Declaring the javadoc contract and Exemplars in your interfaces is a nice approach for contract-driven designs I think.
One last thing, even if you don’t use Exemplars, you might find Sureassert useful for the continuous testing and real-time coverage reporting support for your JUnit tests. Please feel free to give it a try, I’ll try to offer help on the forums!
Post a comment