English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Scene graph generation (SGG) aims to predict graph-structured descriptions ofinput images, in the form of objects and relationships between them. This taskis becoming increasingly useful for progress at the interface of vision andlanguage. Here, it is important - yet challenging - to perform well on novel(zero-shot) or rare (few-shot) compositions of objects and relationships. Inthis paper, we identify two key issues that limit such generalization. Firstly,we show that the standard loss used in this task is unintentionally a functionof scene graph density. This leads to the neglect of individual edges in largesparse graphs during training, even though these contain diverse few-shotexamples that are important for generalization. Secondly, the frequency ofrelationships can create a strong bias in this task, such that a blind modelpredicting the most frequent relationship achieves good performance.Consequently, some state-of-the-art models exploit this bias to improveresults. We show that such models can suffer the most in their ability togeneralize to rare compositions, evaluating two different models on the VisualGenome dataset and its more recent, improved version, GQA. To address theseissues, we introduce a density-normalized edge loss, which provides more than atwo-fold improvement in certain generalization metrics. Compared to other worksin this direction, our enhancements require only a few lines of code and noadded computational cost. We also highlight the difficulty of accuratelyevaluating models using existing metrics, especially on zero/few shots, andintroduce a novel weighted metric.

Graph Density-Aware Losses for Novel Compositions in Scene Graph Generation