Premium
Letting Evidence Speak for Itself: Measuring Confidence in Mechanisms
Author(s) -
Befani Barbara,
D'Errico Stefano
Publication year - 2020
Publication title -
new directions for evaluation
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.374
H-Index - 40
eISSN - 1534-875X
pISSN - 1097-6736
DOI - 10.1002/ev.20420
Subject(s) - credibility , empirical evidence , set (abstract data type) , mechanism (biology) , general partnership , process (computing) , falsifiability , quality (philosophy) , product (mathematics) , cognition , bayes' theorem , measure (data warehouse) , computer science , psychology , epistemology , economics , bayesian probability , artificial intelligence , data mining , philosophy , geometry , mathematics , finance , neuroscience , programming language , operating system
This chapter argues that the credibility of causal mechanisms can be greatly increased by formulating them as statements that are both empirically falsifiable and empirically confirmable. Whether statements can be so depends on the potential availability of the relevant evidence (e.g., no evidence exists that can prove or disprove the existence of God, but good quality evidence is potentially available in many other cases). The Bayes formula can be used to measure the extent to which a given set of empirical observations supports or weakens the belief that a causal mechanism exists. With this approach, confidence in the existence of a mechanism is increased or decreased through a process that can be open, transparent, and shared with the public or groups of stakeholders, reducing cognitive biases, and improving internal validity and consensus around the existence of given mechanisms. The approach is showcased in the evaluation of a learning partnership whereby a knowledge product released by a research organization influenced policy at the municipal level.