English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent large language models (LLMs) have shown indications of mathematicalreasoning ability. However it has not been clear how they would fare on morechallenging competition-level problems. And while self-generated verbalizationsof intermediate reasoning steps (i.e., chain-of-thought prompting) have beenshown to be helpful, whether LLMs can make use of helpful side information suchas problem-specific hints has not been investigated before. In this paper, wepropose a challenging benchmark dataset for enabling such analyses. The Conceptand Hint-Annotated Math Problems (CHAMP) consists of high school mathcompetition problems, annotated with concepts, or general math facts, andhints, or problem-specific tricks. These annotations allow us to explore theeffects of additional information, such as relevant hints, misleading concepts,or related problems. This benchmark is difficult, with the best model onlyscoring 58.1% in standard settings. With concepts and hints, performancesometimes improves, indicating that some models can make use of such sideinformation. We further annotate model-generated solutions for theircorrectness. Using this corpus, we find that models often arrive at the correctfinal answer through wrong reasoning steps. In addition, we test whether modelsare able to verify these solutions, and find that most models struggle. Thedataset and code are available on the project website.

CHAMP: A Competition-level Dataset for Fine-Grained Analyses of LLMs' Mathematical Reasoning Capabilities