English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Vision-based segmentation of the robotic tool during robot-assisted surgeryenables downstream applications, such as augmented reality feedback, whileallowing for inaccuracies in robot kinematics. With the introduction of deeplearning, many methods were presented to solve instrument segmentation directlyand solely from images. While these approaches made remarkable progress onbenchmark datasets, fundamental challenges pertaining to their robustnessremain. We present CaRTS, a causality-driven robot tool segmentation algorithm,that is designed based on a complementary causal model of the robot toolsegmentation task. Rather than directly inferring segmentation masks fromobserved images, CaRTS iteratively aligns tool models with image observationsby updating the initially incorrect robot kinematic parameters through forwardkinematics and differentiable rendering to optimize image feature similarityend-to-end. We benchmark CaRTS with competing techniques on both synthetic aswell as real data from the dVRK, generated in precisely controlled scenarios toallow for counterfactual synthesis. On training-domain test data, CaRTSachieves a Dice score of 93.4 that is preserved well (Dice score of 91.8) whentested on counterfactually altered test data, exhibiting low brightness, smoke,blood, and altered background patterns. This compares favorably to Dice scoresof 95.0 and 86.7, respectively, of the SOTA image-based method. Future workwill involve accelerating CaRTS to achieve video framerate and estimating theimpact occlusion has in practice. Despite these limitations, our results arepromising: In addition to achieving high segmentation accuracy, CaRTS providesestimates of the true robot kinematics, which may benefit applications such asforce estimation. Code is available at: https://github.com/hding2455/CaRTS

CaRTS: Causality-driven Robot Tool Segmentation from Vision and Kinematics Data