English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Grounded Situation Recognition (GSR) is capable of recognizing andinterpreting visual scenes in a contextually intuitive way, yielding salientactivities (verbs) and the involved entities (roles) depicted in images. Inthis work, we focus on the application of GSR in assisting people with visualimpairments (PVI). However, precise localization information of detectedobjects is often required to navigate their surroundings confidently and makeinformed decisions. For the first time, we propose an Open Scene Understanding(OpenSU) system that aims to generate pixel-wise dense segmentation masks ofinvolved entities instead of bounding boxes. Specifically, we build our OpenSUsystem on top of GSR by additionally adopting an efficient Segment AnythingModel (SAM). Furthermore, to enhance the feature extraction and interactionbetween the encoder-decoder structure, we construct our OpenSU system using asolid pure transformer backbone to improve the performance of GSR. In order toaccelerate the convergence, we replace all the activation functions within theGSR decoders with GELU, thereby reducing the training duration. In quantitativeanalysis, our model achieves state-of-the-art performance on the SWiG dataset.Moreover, through field testing on dedicated assistive technology datasets andapplication demonstrations, the proposed OpenSU system can be used to enhancescene understanding and facilitate the independent mobility of people withvisual impairments. Our code will be available athttps://github.com/RuipingL/OpenSU.

Open Scene Understanding: Grounded Situation Recognition Meets Segment Anything for Helping People with Visual Impairments