English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose a new benchmark evaluating the performance of multimodal largelanguage models on rebus puzzles. The dataset covers 333 original examples ofimage-based wordplay, cluing 13 categories such as movies, composers, majorcities, and food. To achieve good performance on the benchmark of identifyingthe clued word or phrase, models must combine image recognition and stringmanipulation with hypothesis testing, multi-step reasoning, and anunderstanding of human cognition, making for a complex, multimodal evaluationof capabilities. We find that proprietary models such as GPT-4V and Gemini Prosignificantly outperform all other tested models. However, even the best modelhas a final accuracy of just 24%, highlighting the need for substantialimprovements in reasoning. Further, models rarely understand all parts of apuzzle, and are almost always incapable of retroactively explaining the correctanswer. Our benchmark can therefore be used to identify major shortcomings inthe knowledge and reasoning of multimodal large language models.

REBUS: A Robust Evaluation Benchmark of Understanding Symbols