English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We introduce KnowledgeMath, a novel benchmark designed to evaluate LLMs'capabilities in applying financial knowledge to solve complex math wordproblems. Compared to prior works, this study features three core advancements.First, KnowledgeMath includes 1,259 problems with a hybrid of textual andtabular content and require college-level knowledge in the finance domain foreffective resolution. Second, we provide expert-annotated, detailed solutionreferences in Python program format, ensuring a high-quality benchmark for LLMassessment. Finally, we evaluate a wide spectrum of 14 LLMs with differentprompting strategies like Chain-of-Thoughts and Program-of-Thoughts. Thecurrent best-performing system (i.e., GPT-4 with Program-of-Thoughts) achievesonly 45.4% accuracy, leaving substantial room for improvement. Whileknowledge-augmented LLMs can improve the performance (e.g., from 23.9% to 32.0%for GPT-3.5), it is still significantly lower the estimated human expertperformance of 94%. We believe that KnowledgeMath can facilitate futureresearch on domain-specific knowledge retrieval and augmentation into the mathword problem-solving process. We will release the benchmark and code athttps://github.com/yale-nlp/KnowledgeMath.

KnowledgeMath: Knowledge-Intensive Math Word Problem Solving in Finance Domains