English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

In this paper, we propose EventBind, a novel and effective framework thatunleashes the potential of vision-language models (VLMs) for event-basedrecognition to compensate for the lack of large-scale event-based datasets. Inparticular, due to the distinct modality gap with the image-text data and thelack of large-scale datasets, learning a common representation space forimages, texts, and events is non-trivial.Intuitively, we need to address twokey challenges: 1) how to generalize CLIP's visual encoder to event data whilefully leveraging events' unique properties, e.g., sparsity and high temporalresolution; 2) how to effectively align the multi-modal embeddings, i.e.,image, text, and events. Accordingly, we first introduce a novel event encoderthat subtly models the temporal information from events and meanwhile,generates event prompts for modality bridging. We then design a text encoderthat generates content prompts and utilizes hybrid text prompts to enhanceEventBind's generalization ability across diverse datasets.With the proposedevent encoder, text encoder, and image encoder, a novel Hierarchical TripleContrastive Alignment (HTCA) module is introduced to jointly optimize thecorrelation and enable efficient knowledge transfer among the three modalities.We evaluate various settings, including fine-tuning and few-shot on threebenchmarks, and our EventBind achieves new state-of-the-art accuracy comparedwith the previous methods, such as on N-Caltech101 (+5.34% and +1.70%) andN-Imagenet (+5.65% and +1.99%) with fine-tuning and 20-shot settings,respectively. Moreover, our EventBind can be flexibly extended to the eventretrieval task using text or image queries, showing plausible performance.Project page:https://vlislab22.github.io/EventBind/.

EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding