English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Image-grounded dialogue systems benefit greatly from integrating visualinformation, resulting in high-quality response generation. However, currentmodels struggle to effectively utilize such information in zero-resourcescenarios, mainly due to the disparity between image and text modalities. Toovercome this challenge, we propose an innovative multimodal framework, calledZRIGF, which assimilates image-grounded information for dialogue generation inzero-resource situations. ZRIGF implements a two-stage learning strategy,comprising contrastive pre-training and generative pre-training. Contrastivepre-training includes a text-image matching module that maps images and textsinto a unified encoded vector space, along with a text-assisted masked imagemodeling module that preserves pre-training visual features and fosters furthermultimodal feature alignment. Generative pre-training employs a multimodalfusion module and an information transfer module to produce insightfulresponses based on harmonized multimodal representations. Comprehensiveexperiments conducted on both text-based and image-grounded dialogue datasetsdemonstrate ZRIGF's efficacy in generating contextually pertinent andinformative responses. Furthermore, we adopt a fully zero-resource scenario inthe image-grounded dialogue dataset to demonstrate our framework's robustgeneralization capabilities in novel domains. The code is available athttps://github.com/zhangbo-nlp/ZRIGF.

ZRIGF: An Innovative Multimodal Framework for Zero-Resource Image-Grounded Dialogue Generation