English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

How to best integrate linguistic and perceptual processing in multi-modaltasks that involve language and vision is an important open problem. In thiswork, we argue that the common practice of using language in a top-down manner,to direct visual attention over high-level visual features, may not be optimal.We hypothesize that the use of language to also condition the bottom-upprocessing from pixels to high-level features can provide benefits to theoverall performance. To support our claim, we propose a U-Net-based model andperform experiments on two language-vision dense-prediction tasks: referringexpression segmentation and language-guided image colorization. We compareresults where either one or both of the top-down and bottom-up visual branchesare conditioned on language. Our experiments reveal that using language tocontrol the filters for bottom-up visual processing in addition to top-downattention leads to better results on both tasks and achieves competitiveperformance. Our linguistic analysis suggests that bottom-up conditioningimproves segmentation of objects especially when input text refers to low-levelvisual concepts. Code is available at https://github.com/ilkerkesen/bvpr.

Modulating Bottom-Up and Top-Down Visual Processing via Language-Conditional Filters