English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large Language Models (LLMs), through their contextualized representations,have been empirically proven to encapsulate syntactic, semantic, word sense,and common-sense knowledge. However, there has been limited exploration oftheir physical reasoning abilities, specifically concerning the crucialattributes for comprehending everyday objects. To address this gap, weintroduce NEWTON, a repository and benchmark for evaluating the physicsreasoning skills of LLMs. Further, to enable domain-specific adaptation of thisbenchmark, we present a pipeline to enable researchers to generate a variant ofthis benchmark that has been customized to the objects and attributes relevantfor their application. The NEWTON repository comprises a collection of 2800object-attribute pairs, providing the foundation for generating infinite-scaleassessment templates. The NEWTON benchmark consists of 160K QA questions,curated using the NEWTON repository to investigate the physical reasoningcapabilities of several mainstream language models across foundational,explicit, and implicit reasoning tasks. Through extensive empirical analysis,our results highlight the capabilities of LLMs for physical reasoning. We findthat LLMs like GPT-4 demonstrate strong reasoning capabilities inscenario-based tasks but exhibit less consistency in object-attribute reasoningcompared to humans (50% vs. 84%). Furthermore, the NEWTON platform demonstratesits potential for evaluating and enhancing language models, paving the way fortheir integration into physically grounded settings, such as roboticmanipulation. Project site: https://newtonreasoning.github.io

NEWTON: Are Large Language Models Capable of Physical Reasoning?