Posts Tagged code quality

Misapprehension 9: It is sufficient to derive the test cases for 100% coverage from the code

Certainly it is possible to derive test cases from the existing source code, with the aim to get 100% coverage by execution of these test cases. There are even tools that do that automatically for you. But you need to be aware of two things thereby:
(1) You cannot detect omissions! From test cases that were derived from the code you will never find out that a part of the code is missing. Think of a check for a maximal value that was forgotten to be implemented. It is simply not possible to derive a test case from the code that will reveal that a piece of the code is not there. If you postulate your code to be complete, the test statement is doubtful, even if you have reached 100% coverage.

(2) The actual result of automatically derived test cases needs to be evaluated against the expected result. If you neglect this you will detect only fatal problems of your test object, e.g. a crash of your test object. If you simply accept the actual results from the execution of the given code, the relevance of the test is strongly reduced. Let’s take as an example for an undetected omission the sinus function from the precedent misapprehension. We ran a test case and got 100%. But we did not detect that the code actually missed some lines.

The omission (in red) was not detected by measuring code coverage

The omission (in red) was not detected by measuring code coverage

Excursus: What is the rationale of the added code? The added code makes the calculation more accurate, because, due to the fixed number of iterations of the for-loop, the result of the calculation becomes more inaccurate the higher the input value is. This is remedied by the added code, which makes use of the fact that sin(x°+360°) = sin(x°). It could well be the case that the specification for the sinus function states to pre-process the input value to get a value below 360° in any case, but the developer of the function has forgotten to implement the necessary code. The function works in spite of this, but not as accurate as it should.
An again: If we would run some test cases on the code in the figure above, and if we would achieve 100% coverage in doing so: Is the code in the figure above complete? I guess not: Consider input values below -360°, e.g. -400°.

, ,

No Comments

Misapprehension 8: Code Coverage measures the quality of your test cases

Not always. Let’s take as an example a function that calculates the sinus function:

Is 100% coverage enough for that function?

Is 100% coverage enough for that function?

For the function in the figure above, a single test case, e.g. with an input value of 0 for the variable x_deg is sufficient to reach 100% branch/decision coverage, 100% MC/DC, and 100% MCC. The actual result is 0, what is the expected and correct value. But this single test case neither is error-sensitive nor is a single test case sufficient, even if you reach 100% coverage.

, , ,

No Comments

Misapprehension 7: Code Coverage measures the quality of your code

Not absolutely. You only can say: The higher the percentage of coverage is, the “better” the code is tested with respect to that coverage measure. You might think this splits a too fine hair, but 100% coverage does not mean 100% software quality.

, ,

No Comments