English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Panoptic Narrative Detection (PND) and Segmentation (PNS) are two challengingtasks that involve identifying and locating multiple targets in an imageaccording to a long narrative description. In this paper, we propose a unifiedand effective framework called NICE that can jointly learn these two panopticnarrative recognition tasks. Existing visual grounding tasks use a two-branchparadigm, but applying this directly to PND and PNS can result in predictionconflict due to their intrinsic many-to-many alignment property. To addressthis, we introduce two cascading modules based on the barycenter of the mask,which are Coordinate Guided Aggregation (CGA) and Barycenter DrivenLocalization (BDL), responsible for segmentation and detection, respectively.By linking PNS and PND in series with the barycenter of segmentation as theanchor, our approach naturally aligns the two tasks and allows them tocomplement each other for improved performance. Specifically, CGA provides thebarycenter as a reference for detection, reducing BDL's reliance on a largenumber of candidate boxes. BDL leverages its excellent properties todistinguish different instances, which improves the performance of CGA forsegmentation. Extensive experiments demonstrate that NICE surpasses allexisting methods by a large margin, achieving 4.1% for PND and 2.9% for PNSover the state-of-the-art. These results validate the effectiveness of ourproposed collaborative learning strategy. The project of this work is madepublicly available at https://github.com/Mr-Neko/NICE.

NICE: Improving Panoptic Narrative Detection and Segmentation with Cascading Collaborative Learning