English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large Language Models (LLMs) and Large Multimodal Models (LMMs) exhibitimpressive problem-solving skills in many tasks and domains, but their abilityin mathematical reasoning in visual contexts has not been systematicallystudied. To bridge this gap, we present MathVista, a benchmark designed tocombine challenges from diverse mathematical and visual tasks. It consists of6,141 examples, derived from 28 existing multimodal datasets involvingmathematics and 3 newly created datasets (i.e., IQTest, FunctionQA, andPaperQA). Completing these tasks requires fine-grained, deep visualunderstanding and compositional reasoning, which all state-of-the-artfoundation models find challenging. With MathVista, we have conducted acomprehensive, quantitative evaluation of 12 prominent foundation models. Thebest-performing GPT-4V model achieves an overall accuracy of 49.9%,substantially outperforming Bard, the second-best performer, by 15.1%. Ourin-depth analysis reveals that the superiority of GPT-4V is mainly attributedto its enhanced visual perception and mathematical reasoning. However, GPT-4Vstill falls short of human performance by 10.4%, as it often struggles tounderstand complex figures and perform rigorous reasoning. This significant gapunderscores the critical role that MathVista will play in the development ofgeneral-purpose AI agents capable of tackling mathematically intensive andvisually rich real-world tasks. We further explore the new ability ofself-verification, the application of self-consistency, and the interactivechatbot capabilities of GPT-4V, highlighting its promising potential for futureresearch. The project is available at https://mathvista.github.io/.

MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts