English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper introduces an efficient strategy to transform Large LanguageModels (LLMs) into Multi-Modal Large Language Models (MLLMs). Byconceptualizing this transformation as a domain adaptation process, i.e.,transitioning from text understanding to embracing multiple modalities, weintriguingly note that, within each attention block, tuning LayerNorm sufficesto yield strong performance. Moreover, when benchmarked against other tuningapproaches like full parameter finetuning or LoRA, its benefits on efficiencyare substantial. For example, when compared to LoRA on a 13B model scale,performance can be enhanced by an average of over 20% across five multi-modaltasks, and meanwhile, results in a significant reduction of trainableparameters by 41.9% and a decrease in GPU memory usage by 17.6%. On top of thisLayerNorm strategy, we showcase that selectively tuning only withconversational data can improve efficiency further. Beyond these empiricaloutcomes, we provide a comprehensive analysis to explore the role of LayerNormin adapting LLMs to the multi-modal domain and improving the expressive powerof the model.

Tuning LayerNorm in Attention: Towards Efficient Multi-Modal LLM Finetuning