Premium
Research Review: Test–retest reliability of standardized diagnostic interviews to assess child and adolescent psychiatric disorders: a systematic review and meta‐analysis
Author(s) -
Duncan Laura,
Comeau Jinette,
Wang Li,
Vitoroulis Irene,
Boyle Michael H.,
Bennett Kathryn
Publication year - 2019
Publication title -
journal of child psychology and psychiatry
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.652
H-Index - 211
eISSN - 1469-7610
pISSN - 0021-9630
DOI - 10.1111/jcpp.12876
Subject(s) - meta analysis , psychology , reliability (semiconductor) , clinical psychology , systematic review , kappa , sample size determination , test (biology) , psychiatry , medline , medicine , statistics , paleontology , power (physics) , linguistics , physics , philosophy , mathematics , quantum mechanics , political science , law , biology
Background A better understanding of factors contributing to the observed variability in estimates of test–retest reliability in published studies on standardized diagnostic interviews ( SDI ) is needed. The objectives of this systematic review and meta‐analysis were to estimate the pooled test–retest reliability for parent and youth assessments of seven common disorders, and to examine sources of between‐study heterogeneity in reliability. Methods Following a systematic review of the literature, multilevel random effects meta‐analyses were used to analyse 202 reliability estimates (Cohen's kappa = ҡ) from 31 eligible studies and 5,369 assessments of 3,344 children and youth. Results Pooled reliability was moderate at ҡ = .58 ( CI 95% 0.53–0.63) and between‐study heterogeneity was substantial ( Q = 2,063 ( df = 201), p < .001 and I 2 = 79%). In subgroup analysis, reliability varied across informants for specific types of psychiatric disorder (ҡ = .53–.69 for parent vs. ҡ = .39–.68 for youth) with estimates significantly higher for parents on attention deficit hyperactivity disorder, oppositional defiant disorder and the broad groupings of externalizing and any disorder. Reliability was also significantly higher in studies with indicators of poor or fair study methodology quality (sample size <50, retest interval <7 days). Conclusions Our findings raise important questions about the meaningfulness of published evidence on the test–retest reliability of SDI s and the usefulness of these tools in both clinical and research contexts. Potential remedies include the introduction of standardized study and reporting requirements for reliability studies, and exploration of other approaches to assessing and classifying child and adolescent psychiatric disorder.