Deep learning‐based X‐ray inpainting for improving spinal 2D‐3D registration

Background Two‐dimensional (2D)‐3D registration is challenging in the presence of implant projections on intraoperative images, which can limit the registration capture range. Here, we investigate the use of deep‐learning‐based inpainting for removing implant projections from the X‐rays to improve the registration performance. Methods We trained deep‐learning‐based inpainting models that can fill in the implant projections on X‐rays. Clinical datasets were collected to evaluate the inpainting based on six image similarity measures. The effect of X‐ray inpainting on capture range of 2D‐3D registration was also evaluated. Results The X‐ray inpainting significantly improved the similarity between the inpainted images and the ground truth. When applying inpainting before the 2D‐3D registration process, we demonstrated significant recovery of the capture range by up to 85%. Conclusion Applying deep‐learning‐based inpainting on X‐ray images masked by implants can markedly improve the capture range of the associated 2D‐3D registration task.