English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In recent years, the integration of vision and language understanding has ledto significant advancements in artificial intelligence, particularly throughVision-Language Models (VLMs). However, existing VLMs face challenges inhandling real-world applications with complex scenes and multiple objects, aswell as aligning their focus with the diverse attention patterns of humanusers. In this paper, we introduce gaze information, feasibly collected by ARor VR devices, as a proxy for human attention to guide VLMs and propose a novelapproach, Voila-A, for gaze alignment to enhance the interpretability andeffectiveness of these models in real-world applications. First, we collecthundreds of minutes of gaze data to demonstrate that we can mimic human gazemodalities using localized narratives. We then design an automatic dataannotation pipeline utilizing GPT-4 to generate the VOILA-COCO dataset.Additionally, we innovate the Voila Perceiver modules to integrate gazeinformation into VLMs while preserving their pretrained knowledge. We evaluateVoila-A using a hold-out validation set and a newly collected VOILA-GAZETestset, which features real-life scenarios captured with a gaze-trackingdevice. Our experimental results demonstrate that Voila-A significantlyoutperforms several baseline models. By aligning model attention with humangaze patterns, Voila-A paves the way for more intuitive, user-centric VLMs andfosters engaging human-AI interaction across a wide range of applications.

Voila-A: Aligning Vision-Language Models with User's Gaze Attention