Some Common Misperceptions About P Values

A P value <0.05 is perceived by many as the Holy Grail of clinical trials (as with most research in the natural and social sciences). It is greatly sought after because of its (undeserved) power to persuade the clinical community to accept or not accept a new treatment into practice. Yet few, if any, of us know why 0.05 is so sacred. Literature abounds with answers to the question, What is a P value?, and how the value 0.05 was adopted, more or less arbitrarily or subjectively, by R.A. Fisher, in the 1920s. He selected 0.05 partly because of the convenient fact that in a normal distribution, the 5% cutoff falls around the second standard deviation away from the mean.1However, little is written on how 0.05 became the standard by which many clinical trial results have been judged. A commentary2 ponders whether this phenomenon is similar to the results from the monkeys in the stairs experiment, whereby a group of monkeys were placed in a cage with a set of stairs with some fruit at the top. When a monkey went on the steps, blasts of air descended on it as a deterrent. After a while, any monkey that attempted to get on the steps was dissuaded by the group. Eventually, the monkeys were gradually replaced by new monkeys, but the practice of dissuasion continued, even when the deterrent was no longer rendered. In other words, the new monkeys were unaware of the reason why they were not supposed to go up the steps, yet the practice continued.In the following, I first review what a P value is. Then, I address 2 of the many issues regarding P values in clinical trials. The first challenges the conventional need to show P <0.05 to conclude statistical significance …