Comparison of chest X-ray interpretation by Emergency Department clinicians and radiologists in suspected COVID-19 infection: a retrospective cohort study

Objectives: We describe the inter-rater agreement between Emergency Department (ED) clinicians and reporting radiologists in the interpretation of chest X-rays (CXRs) in patients presenting to ED with suspected COVID-19. Methods: We undertook a retrospective cohort study of patients with suspected COVID-19. We compared ED clinicians’ and radiologists’ interpretation of the CXRs according to British Society of Thoracic Imaging (BSTI) guidelines, using the area under the receiver operator curve (ROC area). Results: CXRs of 152 cases with suspected COVID-19 infection were included. Sensitivity and specificity for ‘classic’ COVID-19 CXR findings reported by ED clinician was 84 and 83%, respectively, with a ROC area of 0.84 (95%CI 0.77 to 0.90). Accuracy improved with ED clinicians’ experience, with ROC areas of 0.73 (95%CI 0.45 to 1.00), 0.81 (95%CI 0.73 to 0.89), 1.00 (95%CI 1.00 to 1.00) and 0.90 (95%CI 0.70 to 1.00) for foundation year doctors, senior house officers, higher speciality trainees and ED consultants, respectively ( p < 0.001). Conclusions: ED clinicians demonstrated moderate inter-rater agreement with reporting radiologists according to the BSTI COVID-19 classifications. The improvement in accuracy with ED clinician experience suggests training of junior ED clinicians in the interpretation of COVID-19 related CXRs might be beneficial. Large-scale survey studies might be useful in the further evaluation of this topic. Advances in knowledge: This is the first study to examine inter-rater agreement between ED clinicians and radiologists in regards to COVID-19 CXR interpretation. Further service configurations such as 24-hr hot reporting of CXRs can be guided by these data, as well as an ongoing, nationwide follow-up study.