Premium
THE MEANING OF FAILED REPLICATIONS: A REVIEW AND PROPOSAL
Author(s) -
Clemens Michael A.
Publication year - 2017
Publication title -
journal of economic surveys
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.657
H-Index - 92
eISSN - 1467-6419
pISSN - 0950-0804
DOI - 10.1111/joes.12139
Subject(s) - replication (statistics) , terminology , meaning (existential) , incentive , computer science , positive economics , economics , psychology , microeconomics , statistics , linguistics , mathematics , philosophy , psychotherapist
The welcome rise of replication tests in economics has not been accompanied by a consensus standard for determining what constitutes a replication . A discrepant replication, in current usage of the term, can signal anything from an unremarkable disagreement over methods to scientific incompetence or misconduct. This paper proposes a standard for classifying one study as a replication of some other study. It is a standard that places the burden of proof on a study to demonstrate that it should have obtained identical results to the original, a conservative standard that is already used implicitly by many researchers. It contrasts this standard with decades of unsuccessful attempts to harmonize terminology, and argues that many prominent results described as replication tests should not be described as such. Adopting a conservative standard like this one can improve incentives for researchers, encouraging more and better replication tests.