English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Can a Large Language Model (LLM) solve simple abstract reasoning problems? Weexplore this broad question through a systematic analysis of GPT on theAbstraction and Reasoning Corpus (ARC), a representative benchmark of abstractreasoning ability from limited examples in which solutions require some "coreknowledge" of concepts such as objects, goal states, counting, and basicgeometry. GPT-4 solves only 13/50 of the most straightforward ARC tasks whenusing textual encodings for their two-dimensional input-output grids. Ourfailure analysis reveals that GPT-4's capacity to identify objects and reasonabout them is significantly influenced by the sequential nature of the textthat represents an object within a text encoding of a task. To test thishypothesis, we design a new benchmark, the 1D-ARC, which consists ofone-dimensional (array-like) tasks that are more conducive to GPT-basedreasoning, and where it indeed performs better than on the (2D) ARC. Toalleviate this issue, we propose an object-based representation that isobtained through an external tool, resulting in nearly doubling the performanceon solved ARC tasks and near-perfect scores on the easier 1D-ARC. Although thestate-of-the-art GPT-4 is unable to "reason" perfectly within non-languagedomains such as the 1D-ARC or a simple ARC subset, our study reveals that theuse of object-based representations can significantly improve its reasoningability. Visualizations, GPT logs, and data are available athttps://khalil-research.github.io/LLM4ARC.

LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations