English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we present a novel generative task: joint scene graph - imagegeneration. While previous works have explored image generation conditioned onscene graphs or layouts, our task is distinctive and important as it involvesgenerating scene graphs themselves unconditionally from noise, enablingefficient and interpretable control for image generation. Our task ischallenging, requiring the generation of plausible scene graphs withheterogeneous attributes for nodes (objects) and edges (relations amongobjects), including continuous object bounding boxes and discrete object andrelation categories. We introduce a novel diffusion model, DiffuseSG, thatjointly models the adjacency matrix along with heterogeneous node and edgeattributes. We explore various types of encodings for the categorical data,relaxing it into a continuous space. With a graph transformer being thedenoiser, DiffuseSG successively denoises the scene graph representation in acontinuous space and discretizes the final representation to generate the cleanscene graph. Additionally, we introduce an IoU regularization to enhance theempirical performance. Our model significantly outperforms existing methods inscene graph generation on the Visual Genome and COCO-Stuff datasets, both onstandard and newly introduced metrics that better capture the problemcomplexity. Moreover, we demonstrate the additional benefits of our model intwo downstream applications: 1) excelling in a series of scene graph completiontasks, and 2) improving scene graph detection models by using extra trainingsamples generated from DiffuseSG.

Joint Generative Modeling of Scene Graphs and Images via Diffusion Models