English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we extend scene understanding to include that of human sketch.The result is a complete trilogy of scene representation from three diverse andcomplementary modalities -- sketch, photo, and text. Instead of learning arigid three-way embedding and be done with it, we focus on learning a flexiblejoint embedding that fully supports the ``optionality" that thiscomplementarity brings. Our embedding supports optionality on two axes: (i)optionality across modalities -- use any combination of modalities as query fordownstream tasks like retrieval, (ii) optionality across tasks --simultaneously utilising the embedding for either discriminative (e.g.,retrieval) or generative tasks (e.g., captioning). This provides flexibility toend-users by exploiting the best of each modality, therefore serving the verypurpose behind our proposal of a trilogy in the first place. First, acombination of information-bottleneck and conditional invertible neuralnetworks disentangle the modality-specific component from modality-agnostic insketch, photo, and text. Second, the modality-agnostic instances from sketch,photo, and text are synergised using a modified cross-attention. Once learned,we show our embedding can accommodate a multi-facet of scene-related tasks,including those enabled for the first time by the inclusion of sketch, allwithout any task-specific modifications. Project Page:\url{http://www.pinakinathc.me/scenetrilogy}

SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Text