Commentary: The P-value, devalued

Of the many honorifics bestowed on the articles in this historical series, it is doubtful that any have had applied the best—funny. The rhetorical zest and smiling outrage that Joseph Berkson brings to his puncturing of the quasi-religious precepts of traditional statistics in his classic article 1 recalls for me a public debate I witnessed in the 1980s between a highly respected statistician and a surgeon clinical-trialist. It was a debate on issues related to the adjustment of P-values in clinical trials, and what I remember best was the entrance of the physician in full surgical regalia; green operating scrubs, face mask, shoe covers, the whole bit. Playing effectively the role of the ‘aw-shucks, I’m just a country doc who don’t know nuthin’ ‘bout statistics’ he parodied traditional statistical precepts so effectively, contrasting them unfavourably with common-sense judgements, that the statistician, however meritorious his rebuttal may have been, was left sputtering, helplessly pounding the lectern. So it seems with this commentary, which asks in an innocent yet seemingly unanswerable way, ‘If the population [of people] is not human, what is it?’ This is the leading edge of an attack on Fisher’s P-value which should still be required reading for all students of epidemiology and biostatistics today. The commentary shows us several things. First, it demonstrates just how old are some current criticisms, often presented as enlightened insights from a modern era. His first sentence has almost a nostalgic quality that looks surprising over 60 years later, ‘There was a time when we did not talk about tests of significance; we simply did them.’ These words described the future as much as the pre-1942 past. Second, although it may not be immediately obvious, the argument presented here is closely related to ones that underlie modern recommendations to use CI and even Bayesian methods in lieu of P-values in biomedical research. Third, Berkson makes important distinctions between hypothesis testing and significance tests that continue to be ignored today. Fourth, and perhaps most subtly, he brings in a notion of ‘evidence’, a positive, relative concept that is critical to have on the table as separate and distinct from the P-value. And finally, he provides modern statisticians with a model for how to communicate technical concepts to applied users in an accessible and lively way. All that said, it must be admitted that Berkson’s critique is frustratingly incomplete. While he offers a scathing critique of the P-value, and shows us how standard interpretations contravene scientific intuition (grounded mainly in appeals to common sense) he does not offer a real alternative. He does call for more research, particularly into the meaning of what he calls ‘middle P’s’. It is in this gap that I will spend most of my time in this commentary; linking his insights with the ‘further research’ that indeed occurred over the succeeding 60 years.