English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent advancements in text-only large language models (LLMs) havehighlighted the benefit of in-context learning for adapting to new tasks with afew demonstrations. However, extending in-context learning to largevision-language models (VLMs) using a huge amount of naturalisticvision-language data has shown limited success, particularly for egocentricvideos, due to high data collection costs. We propose a novel training method$\mathbb{E}$fficient $\mathbb{I}$n-context $\mathbb{L}$earning on$\mathbb{E}$gocentric $\mathbb{V}$ideos ($\mathbb{EILEV}$), which elicitsin-context learning in VLMs for egocentric videos without requiring massive,naturalistic egocentric video datasets. $\mathbb{EILEV}$ involves architecturaland training data adaptations to allow the model to process contextsinterleaved with video clips and narrations, sampling of in-context exampleswith clusters of similar verbs and nouns, use of data with skewed marginaldistributions with a long tail of infrequent verbs and nouns, as well ashomonyms and synonyms. Our evaluations show that $\mathbb{EILEV}$-trainedmodels outperform larger VLMs trained on a huge amount of naturalistic data inin-context learning. Furthermore, they can generalize to not onlyout-of-distribution, but also novel, rare egocentric videos and texts viain-context learning, demonstrating potential for applications requiringcost-effective training, and rapid post-deployment adaptability. Our code anddemo are available at \url{https://github.com/yukw777/EILEV}.

Efficient In-Context Learning in Vision-Language Models for Egocentric Videos