English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this work, we focus on open vocabulary instance segmentation to expand asegmentation model to classify and segment instance-level novel categories.Previous approaches have relied on massive caption datasets and complexpipelines to establish one-to-one mappings between image regions and words incaptions. However, such methods build noisy supervision by matching non-visiblewords to image regions, such as adjectives and verbs. Meanwhile, context wordsare also important for inferring the existence of novel objects as they showhigh inter-correlations with novel categories. To overcome these limitations,we devise a joint \textbf{Caption Grounding and Generation (CGG)} framework,which incorporates a novel grounding loss that only focuses on matching objectnouns to improve learning efficiency. We also introduce a caption generationhead that enables additional supervision and contextual modeling as acomplementation to the grounding loss. Our analysis and results demonstratethat grounding and generation components complement each other, significantlyenhancing the segmentation performance for novel classes. Experiments on theCOCO dataset with two settings: Open Vocabulary Instance Segmentation (OVIS)and Open Set Panoptic Segmentation (OSPS) demonstrate the superiority of theCGG. Specifically, CGG achieves a substantial improvement of 6.8% mAP for novelclasses without extra data on the OVIS task and 15% PQ improvements for novelclasses on the OSPS benchmark.

Betrayed by Captions: Joint Caption Grounding and Generation for Open Vocabulary Instance Segmentation