Open AccessA head-to-head comparison of the intra- and interobserver agreement of COVID-RADS and CO-RADS grading systems in a population with high estimated prevalence of COVID-19
Author(s) -
Nikita Sushentsev,
Vlad Bura,
M. Kotnik,
Grigoriy Shiryaev,
Iztok Caglič,
Jonathan R. WeirMcCall,
Tristan Barrett
Publication year - 2020
Publication title -
bjr|open
Language(s) - English
Resource type - Journals
ISSN - 2513-9878
DOI - 10.1259/bjro.20200053
Subject(s) - medicine , covid-19 , confidence interval , bi rads , grading (engineering) , nuclear medicine , population , radiology , kappa , cancer , pathology , mammography , disease , environmental health , breast cancer , outbreak , infectious disease (medical specialty) , linguistics , philosophy , civil engineering , engineering
Objective: To evaluate the inter- and intraobserver agreement of COVID-RADS and CO-RADS reporting systems among differently experienced radiologists in a population with high estimated prevalence of COVID-19. Methods and materials: Chest CT scans of patients with clinically–epidemiologically diagnosed COVID-19 were retrieved from an open-source MosMedData data set, randomised, and independently assigned COVID-RADS and CO-RADS grades by an abdominal radiology fellow, thoracic imaging fellow and a consultant cardiothoracic radiologist. The inter- and intraobserver agreement of the two systems were assessed using the Fleiss’ and Cohen’s κ coefficients, respectively. Results: A total of 200 studies were included in the analysis. Both systems demonstrated moderate interobserver agreement, with κ values of 0.51 [95% confidence interval (CI): 0.46–0.56] and 0.55 (95% CI: 0.50–0.59) for COVID-RADS and CO-RADS, respectively. When COVID-RADS and CO-RADS grades were dichotomised at cut-off values of 2B and 4 to evaluate the agreement between grades representing different levels of clinical suspicion for COVID-19, the interobserver agreement became substantial with κ values of 0.74 (95% CI: 0.66–0.82) for COVID-RADS and 0.73 (95% CI: 0.65–0.81) for CO-RADS. The median intraobserver agreement was considerably higher for CO-RADS reaching 0.81 (95% CI: 0.43–0.76) compared with 0.60 (95% CI: 0.43–0.76) of COVID-RADS. Conclusions: COVID-RADS and CO-RADS showed comparable interobserver agreement, which was moderate when grades were compared head-to-head and substantial when grades were dichotomised to better reflect the underlying levels of suspicion for COVID-19. The median intraobserver agreement of CO-RADS was, however, considerably higher compared with COVID-RADS. Advances in knowledge: This paper provides a comprehensive review of the newly introduced COVID-19 chest CT reporting systems, which will help radiologists of all sub-specialties and experience levels make an informed decision on which system to use in their own practice.