Posts Tagged decision coverage

Misapprehension 11: With respect to coverage, it makes no difference how a compound decision is written

Are these two decisions equivalent with respect to coverage?

Are these two decisions equivalent with respect to coverage?

The two compound decisions D1 and D2 in the figure above are functionally equivalent, i.e. all possible input combinations yield the same result. However, in the C/C++ programming language, there are differences with respect to coverage. For instance, to reach 100% Multiple Condition Coverage (MCC) for D1, you need five test cases, whereas for D2, you only need four. This is caused by the incomplete evaluation (aka short-circuit operation) of the C/C++ programming language. As soon as the compiler knows the outcome of the whole decision, evaluation is stopped. Therefore, for D2 only four test cases are possible, which are (or need to be) sufficient for 100% MC/DC.

[BTW: Because the C/C++ compiler always evaluates decisions from left to right, usage of D2 will lead to faster evaluation on average, because the short-circuit operation will occur more often for D2 than for D1.]

, , ,

No Comments

Misapprehension 10: We must reach 100% MC/DC, because then our software is error-free

It is correct, that Modified Condition / Decision Coverage (MC/DC) is one of the more sophisticated coverage measures, which for instance needs to be provided for products on level A (the most safety-critical level) according to DO-178B. However, to reach 100% MC/DC you usually need more test cases (i.e. test effort) than for less demanding measures, e.g. branch coverage. Is this higher testing effort worthwhile? If you have a huge budget for testing, you certainly should consider aiming for 100% MC/DC. Unfortunately, in reality, the budget for testing is often limited. Therefore, you should ask yourself, if you shouldn’t invest the additional effort required to reach 100% MC/DC in other kinds of tests, e.g. in reviews, mutation testing, or static analysis. The rationale for this is that even 100% MC/DC does not guarantee error-free code, because e.g. MC/DC is insensitive against errors in calculations, like any other coverage measure.
However, the author must admit, that he knows from a case in real life, where unit testing was already finished with 100% branch coverage reached. Then, the requirement for 100% MC/DC was raised, which required additional test cases, which actually revealed a programming error. This error would have gone undetected without the objective for 100% MC/DC.

, , , ,

No Comments

Misapprehension 6: Clever programming makes live easy

This seems to be a consequence from the misapprehension above. Unfortunately this is not correct.

Are two test cases sufficient for 100% MC/DC?

Because a clever programmer has extracted the combination of the conditions from the decision of the if-instruction, at a glance only two test cases seem to be necessary to reach 100% modified condition / decision coverage (MC/DC) for the software snippet on the left hand side in the figure above: One test case in which decision evaluates to true and one test case in which decision evaluates to false. However, the right hand side of the figure above reveals that Tessy has recognized the combination of the conditions in the assignment and therefore three test cases are necessary to reach 100% MC/DC. This prevents creative programming to by-pass correct determination of condition coverage.

, , , , ,

No Comments

Misapprehension 2: There is always only one name for a coverage measure

Unfortunately not. Depending on the standard, book, tool, or web site you consult, different names are used for one and the same measure.
For instance, [Spillner] uses the term “branch condition combination testing“; whereas [Liggesmeyer] calls the same measure “multiple condition coverage“.


, , ,

No Comments

Misapprehension 1: 100% can always be reached

Of course not. It is easy to find real-world examples where the structure of the code prevents the software from being executed completely.

Branch Coverage

Figure: An example where 100% branch coverage cannot be reached

The figure above designates a software snippet where 100% branch coverage cannot be reached. This is a real-world example that was simplified.
Why can 100% not be reached? The cause is the missing execution of the “default:” case of the switch instruction. This “default:” case is always present, whether it is programmed explicitly or not.


, , ,

No Comments