z-logo
open-access-imgOpen Access
The Meta-Distribution of Standard P-Values
Author(s) -
Nassim Nicholas Taleb
Publication year - 2016
Publication title -
ssrn electronic journal
Language(s) - English
Resource type - Journals
ISSN - 1556-5068
DOI - 10.2139/ssrn.2834266
Subject(s) - distribution (mathematics) , geography , statistics , environmental science , mathematics , mathematical analysis
We present an exact probability distribution (meta-distribution) for p-values across ensembles of statistically identical phenomena, as well as the distribution of the minimum p-value among m independents tests. We derive the distribution for small samples 2 P-values are shown to be extremely skewed and volatile, regardless of the sample size $n$, and vary greatly across repetitions of exactly same protocols under identical stochastic copies of the phenomenon; such volatility makes the minimum p value diverge significantly from the "true" one. Setting the power is shown to offer little remedy unless sample size is increased markedly or the p-value is lowered by at least one order of magnitude.The formulas allow the investigation of the stability of the reproduction of results and "p-hacking" and other aspects of meta-analysis. From a probabilistic standpoint, neither a p-value of .05 nor a "power" at .9 appear to make the slightest sense.

The content you want is available to Zendy users.

Already have an account? Click here to sign in.
Having issues? You can contact us here
Accelerating Research

Address

John Eccles House
Robert Robinson Avenue,
Oxford Science Park, Oxford
OX4 4GP, United Kingdom