Open AccessThe bumpy road to achieve reliability of clinical profile characteristics in psychosis and related disorders
Author(s) -
Berendsen Steven,
van Tricht Mirjam J.,
Tedja Amy,
Burger Thijs J.,
de Koning Mariken B.,
de Haan Lieuwe
Publication year - 2021
Publication title -
international journal of methods in psychiatric research
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.275
H-Index - 73
eISSN - 1557-0657
pISSN - 1049-8931
DOI - 10.1002/mpr.1858
Subject(s) - intraclass correlation , reliability (semiconductor) , psychology , statistics , clinical psychology , mathematics , psychometrics , power (physics) , physics , quantum mechanics
Objectives Profile characteristics are factors that are relevant for diagnosis, prognosis or treatment. The present study aims to develop a set of clinically relevant profile characteristics. Moreover, our goal is to determine the inter‐rater reliability (IRR) of the selected profile characteristics. Methods Potential profile characteristics were determined by literature review. Assessment of IRR was done by comparing scores on profile characteristics determined by two researchers. We conducted three subsequent studies: (1) assessment of pre‐training IRR, (2) IRR following implementation of an instruction manual, (3) IRR after optimizing scoring methods. IRR was measured with the Intraclass Correlation Coefficient (ICC). Results IRR scores of profile characteristic Illegal activities were high across the three studies (ICC ≥ 0.75). Following training procedures in study 2 and 3, reliability estimates remained low to moderate (ICC < 0.75) for the profile characteristics Support of relatives, Aggression recent and lifetime, substance use and insight recent. IRR scores of the other eight profile characteristics varied from low, moderate to high across studies. Conclusion IRR scores of profile characteristics were highly variable, and mostly inadequate in all three studies. Consequently, further research should focus on specification of severity scores of profile characteristics, optimizing scoring methods and re‐evaluation of IRR.