English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Humans learn language via multi-modal knowledge. However, due to thetext-only pre-training scheme, most existing pre-trained language models (PLMs)are hindered from the multi-modal information.  To inject visual knowledge into PLMs, existing methods incorporate either thetext or image encoder of vision-language models (VLMs) to encode the visualinformation and update all the original parameters of PLMs for knowledgefusion.  In this paper, we propose a new plug-and-play module, X-adapter, to flexiblyleverage the aligned visual and textual knowledge learned in pre-trained VLMsand efficiently inject them into PLMs.  Specifically, we insert X-adapters into PLMs, and only the added parametersare updated during adaptation.  To fully exploit the potential in VLMs, X-adapters consist of twosub-modules, V-expert and T-expert, to fuse VLMs' image and textrepresentations, respectively.  We can opt for activating different sub-modules depending on the downstreamtasks.  Experimental results show that our method can significantly improve theperformance on object-color reasoning and natural language understanding (NLU)tasks compared with PLM baselines.

Towards Versatile and Efficient Visual Knowledge Integration into Pre-trained Language Models with Cross-Modal Adapters