English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose novel evaluations for mathematical reasoning capabilities of LargeLanguage Models (LLMs) based on mathematical misconceptions. Our primaryapproach is to simulate LLMs as a novice learner and an expert tutor, aiming toidentify the incorrect answer to math question resulted from a specificmisconception and to recognize the misconception(s) behind an incorrect answer,respectively. Contrary to traditional LLMs-based mathematical evaluations thatfocus on answering math questions correctly, our approach takes inspirationsfrom principles in educational learning sciences. We explicitly ask LLMs tomimic a novice learner by answering questions in a specific incorrect mannerbased on incomplete knowledge; and to mimic an expert tutor by identifyingmisconception(s) corresponding to an incorrect answer to a question. Usingsimple grade-school math problems, our experiments reveal that, while LLMs caneasily answer these questions correctly, they struggle to identify 1) theincorrect answer corresponding to specific incomplete knowledge(misconceptions); 2) the misconceptions that explain particular incorrectanswers. Our study indicates new opportunities for enhancing LLMs' mathreasoning capabilities, especially on developing robust student simulation andexpert tutoring models in the educational applications such as intelligenttutoring systems.

Novice Learner and Expert Tutor: Evaluating Math Reasoning Abilities of Large Language Models with Misconceptions