English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We introduce HallusionBench, a comprehensive benchmark designed for theevaluation of image-context reasoning. This benchmark presents significantchallenges to advanced large visual-language models (LVLMs), such asGPT-4V(Vision) and LLaVA-1.5, by emphasizing nuanced understanding andinterpretation of visual data. The benchmark comprises 346 images paired with1129 questions, all meticulously crafted by human experts. We introduce a novelstructure for these visual questions designed to establish control groups. Thisstructure enables us to conduct a quantitative analysis of the models' responsetendencies, logical consistency, and various failure modes. In our evaluationon HallusionBench, we benchmarked 13 different models, highlighting a 31.42%question-pair accuracy achieved by the state-of-the-art GPT-4V. Notably, allother evaluated models achieve accuracy below 16%. Moreover, our analysis notonly highlights the observed failure modes, including language hallucinationand visual illusion, but also deepens an understanding of these pitfalls. Ourcomprehensive case studies within HallusionBench shed light on the challengesof hallucination and illusion in LVLMs. Based on these insights, we suggestpotential pathways for their future improvement. The benchmark and codebase canbe accessed at https://github.com/tianyi-lab/HallusionBench.

HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination & Visual Illusion in Large Vision-Language Models