English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While Large Language Models (LLMs) excel in several natural languageprocessing tasks, their size and inaccessibility present challenges forextensive practical application. Previous studies acquire specialized skillsthrough distillation on LLMs, which result in trading generic abilities, calledmodel specialization. As for reasoning ability, chain-of-thought wassynthesized to subsequent distillation. However, due to hallucination,synthetic chain-of-thought from LLMs contains faulty reasoning. These incorrectreasoning steps damage the reasoning capability. To tackle above issues, wepropose Program-aided Distillation (PaD), which distills LLMs to obtainspecialized small models in reasoning tasks. In PaD, we strengthen specializedmodels with program-aided reasoning, and help them overcome faulty reasoningsteps with automated error checking. Experimental results demonstrate that, onthe GSM8K benchmark, a 0.06B model using PaD can not only outperform certainLLMs (e.g., LLaMA), but also achieves a 10% improvement over baselines with asignificantly smaller scale of parameters and data. Data pruning analysisreveals that PaD possesses higher training efficiency.

PaD: Program-aided Distillation Specializes Large Models in Reasoning