English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Multimodal Large Language Models (MLLMs) have recently achieved promisingzero-shot accuracy on visual question answering (VQA) -- a fundamental taskaffecting various downstream applications and domains. Given the greatpotential for the broad use of these models, it is important to investigatetheir limitations in dealing with different image and question properties. Inthis work, we investigate whether MLLMs can perceive details as well as largercomponents in images. In particular, we show that their zero-shot accuracy inanswering visual questions is very sensitive to the size of the visual subjectrelated to the question, declining up to $45.91\%$ with size. Furthermore, weshow that this effect is causal by observing that human visual cropping cansignificantly mitigate their sensitivity to size. To scale up the usefulness ofhuman cropping, we propose ViCrop, a general framework that utilizes automaticvisual cropping to enhance zero-shot VQA of MLLMs. We construct five variantsof ViCrop leveraging either external localization models or the decisionprocess of the given MLLM itself. Our results show that ViCrop improves MLLMs'zero-shot accuracy across different VQA datasets, for example, enhancesBLIP2-T5's performance by $32.23\%$ on the TextVQA test set. To facilitatefurther investigation of MLLMs' behaviors, our code is publicly released.

ViCrop: Perceiving Small Visual Details in Zero-shot Visual Question Answering with Multimodal Large Language Models