Open AccessMarker‐less real‐time intra‐operative camera and hand‐eye calibration procedure for surgical augmented reality
Author(s) -
Kalia Megha,
Mathur Prateek,
Navab Nassir,
Salcudean Septimiu E.
Publication year - 2019
Publication title -
healthcare technology letters
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.45
H-Index - 19
ISSN - 2053-3713
DOI - 10.1049/htl.2019.0094
Subject(s) - computer vision , artificial intelligence , computer science , augmented reality , endoscope , calibration , rendering (computer graphics) , transformation matrix , camera resectioning , offset (computer science) , camera auto calibration , computer graphics (images) , mathematics , surgery , medicine , physics , kinematics , statistics , classical mechanics , programming language
Accurate medical Augmented Reality (AR) rendering requires two calibrations, a camera intrinsic matrix estimation and a hand‐eye transformation. We present a unified, practical, marker‐less, real‐time system to estimate both these transformations during surgery. For camera calibration we perform calibrations at multiple distances from the endoscope, pre‐operatively, to parametrize the camera intrinsic matrix as a function of distance from the endoscope. Then, we retrieve the camera parameters intra‐operatively by estimating the distance of the surgical site from the endoscope in less than 1 s. Unlike in prior work, our method does not require the endoscope to be taken out of the patient; for the hand‐eye calibration, as opposed to conventional methods that require the identification of a marker, we make use of a rendered tool‐tip in 3D. As the surgeon moves the instrument and observes the offset between the actual and the rendered tool‐tip, they can select points of high visual error and manually bring the instrument tip to match the virtual rendered tool tip. To evaluate the hand‐eye calibration, 5 subjects carried out the hand‐eye calibration procedure on a da Vinci robot. Average Target Registration Error of approximately 7mm was achieved with just three data points.