The Effect of Imperfect Error Detection on Reliability AssessmentReport
Measurement of software reliability by life testing involves executing the software on large numbers of test cases and recording the results. The number of failures observed is used to bound the failure probability even if the number of failures observed is zero. Most analyses assume that all failures will be observed but in practice this will rarely be the case. In this paper we examine the effect of imperfect error detection, L‘.e., the situation in which a failure of the software may not be observed. If the conventional analysis associated with life testing is used, the confidence in the bound on the failure probability is optimistic. Our results show that imperfect error detection does not necessarily limit the ability of life testing to bound the probability of failure to the very low values required in critical systems. However, we show that the confidence level associated with a bound on failure probability cannot necessarily be made as high as desired unless very strong assumptions are made about the error detection mechanism. Such assumptions are unlikely to be met in practice, and so life testing is likely to be useful only for situations where very high confidence levels are not required.
Note: Abstract extracted from PDF file via OCR
All rights reserved (no additional license for public reuse)
Ammann, P, S Brilliant, and J Knight. "The Effect of Imperfect Error Detection on Reliability Assessment." University of Virginia Dept. of Computer Science Tech Report (1992).
University of Virginia, Department of Computer Science