English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite the availability of computer-aided simulators and recorded videos ofsurgical procedures, junior residents still heavily rely on experts to answertheir queries. However, expert surgeons are often overloaded with clinical andacademic workloads and limit their time in answering. For this purpose, wedevelop a surgical question-answering system to facilitate robot-assistedsurgical scene and activity understanding from recorded videos. Most of theexisting VQA methods require an object detector and regions based featureextractor to extract visual features and fuse them with the embedded text ofthe question for answer generation. However, (1) surgical object detectionmodel is scarce due to smaller datasets and lack of bounding box annotation;(2) current fusion strategy of heterogeneous modalities like text and image isnaive; (3) the localized answering is missing, which is crucial in complexsurgical scenarios. In this paper, we propose Visual QuestionLocalized-Answering in Robotic Surgery (Surgical-VQLA) to localize the specificsurgical area during the answer prediction. To deal with the fusion of theheterogeneous modalities, we design gated vision-language embedding (GVLE) tobuild input patches for the Language Vision Transformer (LViT) to predict theanswer. To get localization, we add the detection head in parallel with theprediction head of the LViT. We also integrate GIoU loss to boost localizationperformance by preserving the accuracy of the question-answering model. Weannotate two datasets of VQLA by utilizing publicly available surgical videosfrom MICCAI challenges EndoVis-17 and 18. Our validation results suggest thatSurgical-VQLA can better understand the surgical scene and localize thespecific area related to the question-answering. GVLE presents an efficientlanguage-vision embedding technique by showing superior performance over theexisting benchmarks.

Surgical-VQLA: Transformer with Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgery