Study of the reliability of statistical timing analysis for real-time systems

Probabilistic and statistical temporal analyses have been developed as a means of determining the worst-case execution and response times of real-time software for decades. A number of such methods have been proposed in the literature, of which the majority claim to be able to provide worst-case timing scenarios with respect to a given likelihood of a certain value being exceeded. Further, such claims are based on either some estimates associated with a probability, or probability distributions with a certain level of confidence. However, the validity of the claims are very much dependent on a number of factors, such as the achieved samples and the adopted distributions for analysis. This paper is the first one that puts side by side existing state of the art statistical and probabilistic analysis techniques, using the probabilistic analysis as the ground truth in order to asses the applicability and performance of the statistical technique. The evaluation clearly shows that for the experiments performed the approach can identify clear differences between a range of techniques and that these differences can be considered valid based on the trends expected from the academic theory.