Premium
Difficulties in benchmarking ecological null models: an assessment of current methods
Author(s) -
Molina Chai,
Stone Lewi
Publication year - 2020
Publication title -
ecology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.144
H-Index - 294
eISSN - 1939-9170
pISSN - 0012-9658
DOI - 10.1002/ecy.2945
Subject(s) - null model , benchmarking , null hypothesis , statistical hypothesis testing
Identifying species interactions and detecting when ecological communities are structured by them is an important problem in ecology and biogeography. Ecologists have developed specialized statistical hypothesis tests to detect patterns indicative of community‐wide processes in their field data. In this respect, null model approaches have proved particularly popular. The freedom allowed in choosing the null model and statistic to construct a hypothesis test leads to a proliferation of possible hypothesis tests from which ecologists can choose to detect these processes. Here, we point out some serious shortcomings of a popular approach to choosing the best hypothesis for the ecological problem at hand that involves benchmarking different hypothesis tests by assessing their performance on artificially constructed data sets. Terminological errors concerning the use of Type I and Type II errors that underlie these approaches are discussed. We argue that the key benchmarking methods proposed in the literature are not a sound guide for selecting null hypothesis tests, and further, that there is no simple way to benchmark null hypothesis tests. Surprisingly, the basic problems identified here do not appear to have been addressed previously, and these methods are still being used to develop and test new null models and summary statistics, from quantifying community structure (e.g., nestedness and modularity) to analyzing ecological networks.