Open AccessAutomated instrument-tracking for 4D video-rate imaging of ophthalmic surgical maneuvers
Author(s) -
Eric Tang,
Mohamed T. ElHaddad,
Shriji Patel,
Yuankai K. Tao
Publication year - 2022
Publication title -
biomedical optics express
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.362
H-Index - 86
ISSN - 2156-7085
DOI - 10.1364/boe.450814
Subject(s) - computer vision , surgical instrument , computer science , artificial intelligence , tracking (education) , distortion (music) , microsurgery , match moving , medical imaging , visualization , medicine , motion (physics) , surgery , psychology , amplifier , pedagogy , computer network , bandwidth (computing)
Intraoperative image-guidance provides enhanced feedback that facilitates surgical decision-making in a wide variety of medical fields and is especially useful when haptic feedback is limited. In these cases, automated instrument-tracking and localization are essential to guide surgical maneuvers and prevent damage to underlying tissue. However, instrument-tracking is challenging and often confounded by variations in the surgical environment, resulting in a trade-off between accuracy and speed. Ophthalmic microsurgery presents additional challenges due to the nonrigid relationship between instrument motion and instrument deformation inside the eye, image field distortion, image artifacts, and bulk motion due to patient movement and physiological tremor. We present an automated instrument-tracking method by leveraging multimodal imaging and deep-learning to dynamically detect surgical instrument positions and re-center imaging fields for 4D video-rate visualization of ophthalmic surgical maneuvers. We are able to achieve resolution-limited tracking accuracy at varying instrument orientations as well as at extreme instrument speeds and image defocus beyond typical use cases. As proof-of-concept, we perform automated instrument-tracking and 4D imaging of a mock surgical task. Here, we apply our methods for specific applications in ophthalmic microsurgery, but the proposed technologies are broadly applicable for intraoperative image-guidance with high speed and accuracy.