Premium
A Survey of Model Evaluation Approaches With a Tutorial on Hierarchical Bayesian Methods
Author(s) -
Shiffrin Richard M.,
Lee Michael D.,
Kim Woojae,
Wagenmakers EricJan
Publication year - 2008
Publication title -
cognitive science
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.498
H-Index - 114
eISSN - 1551-6709
pISSN - 0364-0213
DOI - 10.1080/03640210802414826
Subject(s) - generalization , computer science , bayesian probability , artificial intelligence , machine learning , key (lock) , hierarchical database model , bayes' theorem , bayesian inference , management science , data mining , mathematics , mathematical analysis , computer security , economics
This article reviews current methods for evaluating models in the cognitive sciences, including theoretically based approaches, such as Bayes factors and minimum description length measures; simulation approaches, including model mimicry evaluations; and practical approaches, such as validation and generalization measures. This article argues that, although often useful in specific settings, most of these approaches are limited in their ability to give a general assessment of models. This article argues that hierarchical methods, generally, and hierarchical Bayesian methods, specifically, can provide a more thorough evaluation of models in the cognitive sciences. This article presents two worked examples of hierarchical Bayesian analyses to demonstrate how the approach addresses key questions of descriptive adequacy, parameter interference, prediction, and generalization in principled and coherent ways.