The replication and reproducibility crises: origins and consequences for studies of ecology and evolution

Watch the VIDEO.There is a widespread discussion around a scientific crisis, resulting from a lack of reproducibility of published scientific studies. This was exemplified by Ioannidis’ 2005 paper “Why most published research findings are false” or the 2015 Open Science Collaboration study assessing reproducibility of psychological science. An often-cited reason for this reproducibility crisis is a fundamental misunderstanding of what statistical methods, and in particular P-values, can achieve. In the context of studies of ecology and evolution, I will show how 1) the pressure for publishing “novel” results, 2) what Gelman has called the “garden of forking paths”, i.e. the fact that published analyses represent only one out of many possible analyses, and 3) the often fruitless dichotomy between a null and alternative hypotheses, has led to the present situation. While scientific progress is dependent of major breakthroughs, we also need to find a better balance between confirmatory research – understanding how known effects vary in size according to the context – and exploratory, non-incremental research – finding new effects.