Posts Tagged software quality. software coverage

Misapprehension 9: It is sufficient to derive the test cases for 100% coverage from the code

Certainly it is possible to derive test cases from the existing source code, with the aim to get 100% coverage by execution of these test cases. There are even tools that do that automatically for you. But you need to be aware of two things thereby:
(1) You cannot detect omissions! From test cases that were derived from the code you will never find out that a part of the code is missing. Think of a check for a maximal value that was forgotten to be implemented. It is simply not possible to derive a test case from the code that will reveal that a piece of the code is not there. If you postulate your code to be complete, the test statement is doubtful, even if you have reached 100% coverage.

(2) The actual result of automatically derived test cases needs to be evaluated against the expected result. If you neglect this you will detect only fatal problems of your test object, e.g. a crash of your test object. If you simply accept the actual results from the execution of the given code, the relevance of the test is strongly reduced. Let’s take as an example for an undetected omission the sinus function from the precedent misapprehension. We ran a test case and got 100%. But we did not detect that the code actually missed some lines.

The omission (in red) was not detected by measuring code coverage

The omission (in red) was not detected by measuring code coverage

Excursus: What is the rationale of the added code? The added code makes the calculation more accurate, because, due to the fixed number of iterations of the for-loop, the result of the calculation becomes more inaccurate the higher the input value is. This is remedied by the added code, which makes use of the fact that sin(x°+360°) = sin(x°). It could well be the case that the specification for the sinus function states to pre-process the input value to get a value below 360° in any case, but the developer of the function has forgotten to implement the necessary code. The function works in spite of this, but not as accurate as it should.
An again: If we would run some test cases on the code in the figure above, and if we would achieve 100% coverage in doing so: Is the code in the figure above complete? I guess not: Consider input values below -360°, e.g. -400°.

, ,

No Comments