Wednesday, January 29, 2014

automatic jUnit test generators - review

This could be scary one. 

When I heard term “automatic jUnit test generator” I was thinking: “no one should go that path”, ”units test should be written by the developers”, “do you want to replace devs?”, “test should be written before code!”.

But I gave it closer look, and actually “closer think” – did I really expect that there is some AI module which will go through code and automatically write tests – if so why there is no AI which automatically write code in first place … [maybe soon ;-)]

Before you send thousands of hate comments, let’s give it quick look what is definition and what should you expect by automatic jUnit test generator:
  • it's everywhere - right click on class in eclipse or intelliJ and you’ll have it - that's point first and should be breakthrough in your thinking about generators ;-)
  • if you want to write jUnit for class A that is dependent on: log, dbDao, jmsHelper, etc... - there are few different schools, but for simplicity let’s say that you'll write mocks for each of them
  • then if you'll have:
    • class B,C dependent on jmsHelper and log
    • class D,E dependent on dbDao and log
    • etc ...
  • and there is team of people working on same code
Result => we will see developer per class / copy-paste pattern.

What if instead standard eclipse jUnit plugin you’ll use tool that where you'll create "mappers/patterns" for each class that will produce that mocks automatically?
Then you'll run "generator" and it will create same structure as eclipse/intelliJ "right click", but with all mocks already copied - no need to create boilerplate code by each developer separately?

And … there will be no real test code there - plain output jUnit just like in "right click".

If I calculate it properly, I spend like 50% of testing time - writing boilerplate code [setting up all mocks, adding standard "patterns", etc...].
To speed up it I usually finish with copy-paste pattern which from other hand I'm extreme oppositionist.

Using generators I'll not to do it again - yes, of course I need to do it once at beginning, maybe some maintenance, but then it will be reused, and whole solution will be consistent.

So, now we should be on same page.

At first I was skeptic if something like that could exist, but ..

I've take 4 generators for review:

  • AgitarOne
  • JTest - Parasoft
  • Randoop
  • CodePro AnalytiX - google
there are probably more - but to speed up process of speeding up process I  also have some time constrains  ;-)

Test scenario:
1.     Choose few most “typical” classes for that project – together easy and more complicated.
2.     Using different tools generate test cases.
3.     Compare generated jUnits from usability perspective (what was generated, how many methods, what amount of code was there,..)  also from percentage of code covered.


  • AgitarOne - quite nice, but is not generating plain jUnits, there is strong dependency on Agitar version of jUnits and if you don’t have license it will not work - so I’ll not take it into consideration [commercial]
  • Randoop - created random "stress" test cases, quite different from others - delivers almost full tests, finds typical corner case scenarios (NPE, OutOfIndex, ..) [opensource]
  • jTest Parasoft - created boilerplate test code using plain jUnit (which is significant plus compared to Agitar) - it works just like standard jUnit eclipse/intelliJ plugin which means one test for one method, no code analysis[commercial] 
  • CodePro AnalytiX - created boilerplate test code using plain jUnit and EasyMock, with static code analysis - created number of test per method depends on "if-cases" which was quite good especially in the case when we go through existing stuff - definitely that will speed up writing, there are other cool tools in the pack, supporting: code coverage, dependency analysis, metrics computation, audit, etc... which also is pretty cool as developer don't need to wait till Sonar give you stats after commits - you can have it on-demand [freeware]

Recommendation / Winner:
CodePro AnalytiX

CodePro is good call - is generating lots of boilerplate code for a developer and save lots of time. Still complex code cannot be completely automated, and developer need to fill gaps - but it is showing you that gaps, suggesting what should be tested.
In different scenario (and I'm thinking also about code reviews) I will give a try to randoop - as is generating whole test, developer  don't need to touch the code (almost ;-) ), and is testing specific corner cases - which you can miss otherwise.

Of course there are some promises that "generators will exercise" code, which at end will look like (i.e.: method that is adding two ints):
public void testAdd_1() throws Exception
     int x=1;
     int y=2;
     int result = Tested.add(x, y);
    // add test code here

Still developer need to go there and:
- add asserts,
- change name of the method to something meaningful,
- add comments,
- [change whatever is specific to your team process/code quality]

I my opinion CodePro is really good replacement for standard eclipse jUnit generator - in worst case (day first) you'll have exactly same output, in best case - you'll need to add only asserts!

Thursday, January 23, 2014

14 funny dev jargon found on code horror

1. Jenga Code

It's when the whole thing collapses after you alter a block of code!

2. Megamoth

It stands for MEGA MOnolithic meTHod, often contained inside a God Object, the Megamoth usually stretches over two screens in height.

Megamoths as large as 2k LOC have been sighted!

3. Reality 101 Failure

This program does exactly what is told, although when it's deployed it turns out that the problem was misunderstood. So basically, it is useless! 

4. Bicrement

Adding 2 to a variable.

5. Banana Banana Banana 

Placeholder text indicating that documentation is in progress or yet to be completed. 

Mostly used since FxCop complains when a public function lacks documentation. 

6. Protoduction

A prototype that eventually ends up in production.

7. Smurf Naming Convention

When almost every class has the same prefix. 

8. Common Law Feature

A bug in the application that has been there so long that it is now part and parcel of the expected functionality, 

User support is required to actually fix it.

9. Hindenbug

A catastrophic data destroying bug. 

Related to Counterbug and Bloombug.

10. Nopping

comes from assembler NOP for no-operation.

When writing something from the POV of an AI, and their internal language has a lot of programming jargon in it. 

11. Doctype Decoration

When web designers add a doctype declaration but don't write a valid markup. 

12. Heisenbug

Bug that disappears or alters its characteristics when an attempt is made to study it.

13. Stringly Typed

A riff on strongly typed. 

Used to describe an implementation that needlessly relies on strings when programmer & refactor friendly options are available. 

14. Smug Report

A bug submitted by a user who thinks he knows a lot more about the system's design than he really does. 

Filled with irrelevant suggestions that are always wrong about what he thinks is causing the problem and how we should fix it. 

Source: Coding Horror

Tuesday, January 7, 2014

JAVA: Common basic style errors.

Nothing new - just to remember (found in some old txt file)

  • classes too long
  • methods too long
  • little or no javadoc
  • no convention for distinguishing between local variable, arguments and fields
  • many empty catch books that suppress exception
  • inappropriate use of multiple return statements
  • using exception to define regular program flow
  • excessive use of the instance of operator
  • using floating point data to represent money
  • preferring arrays over collections
  • not ordering class members by scope