Open AccessA power analysis for fidelity measurement sample size determination.
Author(s) -
Lynne Stokes,
Jill H. Allor
Publication year - 2016
Publication title -
psychological methods
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 6.981
H-Index - 151
eISSN - 1939-1463
pISSN - 1082-989X
DOI - 10.1037/met0000069
Subject(s) - fidelity , sample size determination , computer science , sample (material) , coding (social sciences) , statistical power , population , high fidelity , statistics , measure (data warehouse) , data collection , data mining , mathematics , engineering , medicine , telecommunications , chemistry , environmental health , chromatography , electrical engineering
The importance of assessing fidelity has been emphasized recently with increasingly sophisticated definitions, assessment procedures, and integration of fidelity data into analyses of outcomes. Fidelity is often measured through observation and coding of instructional sessions either live or by video. However, little guidance has been provided about how to determine the number of observations needed to precisely measure fidelity. We propose a practical method for determining a reasonable sample size for fidelity data collection when fidelity assessment requires observation. The proposed methodology is based on consideration of the power of tests of the treatment effect of outcome itself, as well as of the relationship between fidelity and outcome. It makes use of the methodology of probability sampling from a finite population, because the fidelity parameters of interest are estimated over a specific, limited time frame using a sample. For example, consider a fidelity measure defined as the number of minutes of exposure to a treatment curriculum during the 36 weeks of the study. In this case, the finite population is the 36 sessions, the parameter (number of minutes over the entire 36 sessions) is a total, and the sample is the observed sessions. Software for the sample size calculation is provided.