English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Although In-Context Learning (ICL) brings remarkable performance gains toLarge Language Models (LLMs), the improvements remain lower than fine-tuning ondownstream tasks. This paper introduces Multi-Modal In-Context Tuning (MMICT),a novel multi-modal fine-tuning paradigm that boosts multi-modal fine-tuning byfully leveraging the promising ICL capability of multi-modal LLMs (MM-LLMs). Wepropose the Multi-Modal Hub (M-Hub), a unified module that captures variousmulti-modal features according to different inputs and objectives. Based onM-Hub, MMICT enables MM-LLMs to learn from in-context visual-guided textualfeatures and subsequently generate outputs conditioned on the textual-guidedvisual features. Moreover, leveraging the flexibility of M-Hub, we design avariety of in-context demonstrations. Extensive experiments on a diverse rangeof downstream multi-modal tasks demonstrate that MMICT significantlyoutperforms traditional fine-tuning strategy and the vanilla ICT method thatdirectly takes the concatenation of all information from different modalitiesas input.

MMICT: Boosting Multi-Modal Fine-Tuning with In-Context Examples