Monday, July 16, 2012

Java code coverage and thrown exceptions

Recently, I've been working with various Java code coverage tools, such as Emma and JaCoCo.

Generally, these tools are quite powerful and sophisticated.

However, I recently learned of a rather fundamental limitation: they only measure a block as "covered" if that block terminates normally; if the block of code terminates due to a thrown exception, the block is not marked as covered.

The Emma documentation discusses exceptions in sections 2.5, 2.6, and 2.7 of the FAQ:

Thus, EMMA does not attempt to track the success or failure of every single bytecode instruction: instead, it marks a basic block covered when the control reaches its last instruction. A covered basic block is thus guaranteed to have executed without failures at least once in a given coverage run session.

...

the motivation behind this has been that most legitimate code is written so that methods return normally (without throwing an exception).

I'm sure I wasn't the only person unaware of this limitation.

There doesn't appear to be a way around this. So, for the time being, just because your Emma or JaCoCo coverage report indicates that a particular block of code is uncovered, you can't immediately conclude that it isn't invoked at all.

It might have been invoked, but terminated due to a thrown exception.

How is this situation handled in other code coverage tools, for other languages in which exceptions are a first-class part of the language?

No comments:

Post a Comment