English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recently, the development of pre-trained vision language foundation models(VLFMs) has led to remarkable performance in many tasks. However, these modelstend to have strong single-image understanding capability but lack the abilityto understand multiple images. Therefore, they cannot be directly applied tocope with image change understanding (ICU), which requires models to captureactual changes between multiple images and describe them in language. In thispaper, we discover that existing VLFMs perform poorly when applied directly toICU because of the following problems: (1) VLFMs generally learn the globalrepresentation of a single image, while ICU requires capturing nuances betweenmultiple images. (2) The ICU performance of VLFMs is significantly affected byviewpoint variations, which is caused by the altered relationships betweenobjects when viewpoint changes. To address these problems, we propose aViewpoint Integration and Registration method. Concretely, we introduce a fusedadapter image encoder that fine-tunes pre-trained encoders by insertingdesigned trainable adapters and fused adapters, to effectively capture nuancesbetween image pairs. Additionally, a viewpoint registration flow and a semanticemphasizing module are designed to reduce the performance degradation caused byviewpoint variations in the visual and semantic space, respectively.Experimental results on CLEVR-Change and Spot-the-Diff demonstrate that ourmethod achieves state-of-the-art performance in all metrics.

Viewpoint Integration and Registration with Vision Language Foundation Model for Image Change Understanding