Temporal volume flow: an approach to tracking failure recovery

The simultaneous use of pre-segmented CT colonoscopy images and optical colonoscopy images during routine endoscopic procedures provides useful clinical information to the gastroenterologist. Blurry images in the video stream can cause the tracking system to fail during the procedure, due to the endoscope touching the colon wall or a polyp. The ability to recover from such failures is necessary to continually track images, and goes towards building a robust tracking system. Identifying similar images before and after the blurry sequence is central to this task. In this work, we propose a Temporal Volume Flow(TVF) based approach to search for a similar image pair before and after blurry sequences in the optical colonoscopy video. TVF employs nonlinear intensity and gradient constancy models, as well as a discontinuity-preserving smoothness constraint to formulate an energy function; minimizing this function between two temporal volumes before and after the blurry sequence results in an estimate of TVF. A voting approach is then used to determine an image pair with the maximum number of point correspondences. Region flow algorithm10 is applied to the selected image pair to determine camera motion parameters. We applied our algorithm to three optical colonoscopy sequences. The first sequence had 235 images in the ascending colon, and 12 blurry images. The image pair selected by TVF decreases the rotation error of the tracking results using the region flow algorithm. Similar results were observed in the second patient in the descending colon, containing 535 images and 24 blurry images. The third sequence contained 580 images in the descending colon and 172 blurry images. Region flow method failed in this case due to improper image pair selection; using TVF to determine the image pair allowed the system to successfully recover from the blurry sequence.