English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Recent arguments that machine learning (ML) is facing a reproducibility andreplication crisis suggest that some published claims in ML research cannot betaken at face value. These concerns inspire analogies to the replication crisisaffecting the social and medical sciences. They also inspire calls for theintegration of statistical approaches to causal inference and predictivemodeling. A deeper understanding of what reproducibility concerns in supervisedML research have in common with the replication crisis in experimental scienceputs the new concerns in perspective, and helps researchers avoid "the worst ofboth worlds," where ML researchers begin borrowing methodologies fromexplanatory modeling without understanding their limitations and vice versa. Wecontribute a comparative analysis of concerns about inductive learning thatarise in causal attribution as exemplified in psychology versus predictivemodeling as exemplified in ML. We identify themes that re-occur in reformdiscussions, like overreliance on asymptotic theory and non-credible beliefsabout real-world data generating processes. We argue that in both fields,claims from learning are implied to generalize outside the specific environmentstudied (e.g., the input dataset or subject sample, modeling implementation,etc.) but are often impossible to refute due to undisclosed sources of variancein the learning pipeline. In particular, errors being acknowledged in ML exposecracks in long-held beliefs that optimizing predictive accuracy using hugedatasets absolves one from having to consider a true data generating process orformally represent uncertainty in performance claims. We conclude by discussingrisks that arise when sources of errors are misdiagnosed and the need toacknowledge the role of human inductive biases in learning and reform.

The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning