Inference‐specific learning for improved medical image segmentation

Abstract Background Deep learning networks map input data to output predictions by fitting network parameters using training data. However, applying a trained network to new, unseen inference data resembles an interpolation process, which may lead to inaccurate predictions if the training and inference data distributions differ significantly. Purpose This study aims to generally improve the prediction accuracy of deep learning networks on the inference case by bridging the gap between training and inference data. Methods We propose an inference‐specific learning strategy to enhance the network learning process without modifying the network structure. By aligning training data to closely match the specific inference data, we generate an inference‐specific training dataset, enhancing the network optimization around the inference data point for more accurate predictions. Taking medical image auto‐segmentation as an example, we develop an inference‐specific auto‐segmentation framework consisting of initial segmentation learning, inference‐specific training data deformation, and inference‐specific segmentation refinement. The framework is evaluated on public abdominal, head‐neck, and pancreas CT datasets comprising 30, 42, and 210 cases, respectively, for medical image segmentation. Results Experimental results show that our method improves the organ‐averaged mean Dice by 6.2% ( p ‐value = 0.001), 1.5% ( p ‐value = 0.003), and 3.7% ( p ‐value < 0.001) on the three datasets, respectively, with a more notable increase for difficult‐to‐segment organs (such as a 21.7% increase for the gallbladder [ p ‐value = 0.004]). By incorporating organ mask‐based weak supervision into the training data alignment learning, the inference‐specific auto‐segmentation accuracy is generally improved compared with the image intensity‐based alignment. Besides, a moving‐averaged calculation of the inference organ mask during the learning process strengthens both the robustness and accuracy of the final inference segmentation. Conclusions By leveraging inference data during training, the proposed inference‐specific learning strategy consistently improves auto‐segmentation accuracy and holds the potential to be broadly applied for enhanced deep learning decision‐making.