Hypothesis tests with precedence probabilities and precedence‐type tests

Precedence probabilities are important tools in a statistician's toolkit. Precedence probabilities can be defined as the probability of observing single samples from K populations in a particular order. Noting that there are K  ! possible orders of K populations; these K  ! parameters are a useful way to measure the effectiveness of a classifier ( AUC / VUS / HUM ). Receiver operating characteristic ( ROC ) curve/surface/manifold, which can be generated by any classifier leads to calculation of the area under curve ( AUC )/volume under surface ( VUS )/hyper‐volume under manifold ( HUM ) can be approximated by a single precedence probability and can be nonparametrically estimated via rank‐based U‐statistic. Precedence probabilities can also be used to test equality of K  > 2 distribution functions. Hypothesis tests related to both these problems mentioned above are discussed. On the other hand, when we are interested in testing if the K distributions are stochastically ordered, we perform a precedence‐type test. Different nonparametric tests are also discussed in relation to precedence‐type tests. WIREs Comput Stat 2018, 10:e1417. doi: 10.1002/wics.1417 This article is categorized under: Statistical and Graphical Methods of Data Analysis > Nonparametric Methods