English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Multimodal Large Language Models (MLLMs) have demonstrated proficiency inhandling a variety of visual-language tasks. However, current MLLM benchmarksare predominantly designed to evaluate reasoning based on static informationabout a single image, and the ability of modern MLLMs to extrapolate from imagesequences, which is essential for understanding our ever-changing world, hasbeen less investigated. To address this challenge, this paper introducesMementos, a new benchmark designed to assess MLLMs' sequential image reasoningabilities. Mementos features 4,761 diverse image sequences with varyinglengths. We also employ a GPT-4 assisted method to evaluate MLLM reasoningperformance. Through a careful evaluation of nine recent MLLMs on Mementos,including GPT-4V and Gemini, we find that they struggle to accurately describedynamic information about given image sequences, often leading tohallucinations/misrepresentations of objects and their corresponding behaviors.Our quantitative analysis and case studies identify three key factors impactingMLLMs' sequential image reasoning: the correlation between object andbehavioral hallucinations, the influence of cooccurring behaviors, and thecompounding impact of behavioral hallucinations. Our dataset is available athttps://github.com/umd-huang-lab/Mementos.

Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences