English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

This paper proposes Multi-modAl Retrieval model via Visual modulE pLugin(MARVEL), which learns an embedding space for queries and multi-modal documentsto conduct retrieval. MARVEL encodes queries and multi-modal documents with aunified encoder model, which helps to alleviate the modality gap between imagesand texts. Specifically, we enable the image understanding ability of thewell-trained dense retriever, T5-ANCE, by incorporating the visual module'sencoded image features as its inputs. To facilitate the multi-modal retrievaltasks, we build the ClueWeb22-MM dataset based on the ClueWeb22 dataset, whichregards anchor texts as queries, and exacts the related text and imagedocuments from anchor-linked web pages. Our experiments show that MARVELsignificantly outperforms the state-of-the-art methods on the multi-modalretrieval dataset WebQA and ClueWeb22-MM. MARVEL provides an opportunity tobroaden the advantages of text retrieval to the multi-model scenario. Besides,we also illustrate that the language model has the ability to extract imagesemantics and partly map the image features to the input word embedding space.All codes are available at https://github.com/OpenMatch/MARVEL.

MARVEL: Unlocking the Multi-Modal Capability of Dense Retrieval via Visual Module Plugin