DOT main principles

In contrast with most of the testing strategies used today and out of which the overall objective is to fix as many bugs as possible, DOT's main objective is to reduce the risk of system failure during operation. Therefore DOT strategy relies on:

DOT strategy involves 2 distinct phases:

DOT debugging phase

During the debugging phase, the number of test cases that will be performed on a specific feature is optimized by trying to reduce the number of test cases OK that do not increase the feature reliability. Thus, the strategy consists of scheduling first the test cases that are most error-prone and stopping the test activity when the rate of test cases OK becomes too high. This is why the first criteria of prioritization represents the risk that one test case could be NOK.

The above consideration must be adjusted by the fact that, from a reliability point of view, it is not imperative to find the maximum of software defects. The most important is to find the “right” defects, i.e. the defects that are most likely to produce operational problems (high recalls rate due to these defects) or the defects that are severe enough to be worth eliminating, even if their occurrence probability is low. This is why the two other criteria of prioritization represent the criticity of the failure that could occur during one test case execution.

The stopping criterion of this phase is based on the intuitive idea that “when the test campaign is no more able to reveal enough bugs, it is probably time to stop”.

DOT qualification phase

During the qualification phase, MaTeLo is used to generate usage-oriented test cases. The number of test cases to be run is optimized by using equivalence class reduction. The stopping criterion of this phase is reached when the estimated operational reliability is acceptable.