Open AccessAutomatic contouring QA method using a deep learning–based autocontouring system
Author(s) -
Rhee Dong Joo,
Akinfenwa Chidinma P. Anakwenze,
Rigaud Bastien,
Jhingran Anuja,
Cardenas Carlos E.,
Zhang Lifei,
Prajapati Surendra,
Kry Stephen F.,
Brock Kristy K.,
Beadle Beth M.,
Shaw William,
O'Reilly Frederika,
Parkes Jeannette,
Burger Hester,
Fakie Nazia,
Trauernicht Chris,
Simonds Hannah,
Court Laurence E.
Publication year - 2022
Publication title -
journal of applied clinical medical physics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.83
H-Index - 48
ISSN - 1526-9914
DOI - 10.1002/acm2.13647
Subject(s) - contouring , hausdorff distance , artificial intelligence , similarity (geometry) , support vector machine , mathematics , computer science , pattern recognition (psychology) , kernel (algebra) , computer vision , nuclear medicine , medicine , image (mathematics) , computer graphics (images) , combinatorics
Purpose To determine the most accurate similarity metric when using an independent system to verify automatically generated contours. Methods A reference autocontouring system (primary system to create clinical contours) and a verification autocontouring system (secondary system to test the primary contours) were used to generate a pair of 6 female pelvic structures (UteroCervix [uterus + cervix], CTVn [nodal clinical target volume (CTV)], PAN [para‐aortic lymph nodes], bladder, rectum, and kidneys) on 49 CT scans from our institution and 38 from other institutions. Additionally, clinically acceptable and unacceptable contours were manually generated using the 49 internal CT scans. Eleven similarity metrics (volumetric Dice similarity coefficient (DSC), Hausdorff distance, 95% Hausdorff distance, mean surface distance, and surface DSC with tolerances from 1 to 10 mm) were calculated between the reference and the verification autocontours, and between the manually generated and the verification autocontours. A support vector machine (SVM) was used to determine the threshold that separates clinically acceptable and unacceptable contours for each structure. The 11 metrics were investigated individually and in certain combinations. Linear, radial basis function, sigmoid, and polynomial kernels were tested using the combinations of metrics as inputs for the SVM. Results The highest contouring error detection accuracies were 0.91 for the UteroCervix, 0.90 for the CTVn, 0.89 for the PAN, 0.92 for the bladder, 0.95 for the rectum, and 0.97 for the kidneys and were achieved using surface DSCs with a thickness of 1, 2, or 3 mm. The linear kernel was the most accurate and consistent when a combination of metrics was used as an input for the SVM. However, the best model accuracy from the combinations of metrics was not better than the best model accuracy from a surface DSC as an input. Conclusions We distinguished clinically acceptable contours from clinically unacceptable contours with an accuracy higher than 0.9 for the targets and critical structures in patients with cervical cancer; the most accurate similarity metric was surface DSC with a thickness of 1, 2, or 3 mm.