English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Chain-of-thought prompting (e.g., "Let's think step-by-step") primes largelanguage models to verbalize rationalization for their predictions. Whilechain-of-thought can lead to dramatic performance gains, benefits appear toemerge only for sufficiently large models (beyond 50B parameters). We show thatorders-of-magnitude smaller models (125M -- 1.3B parameters) can still benefitfrom chain-of-thought prompting. To achieve this, we introduce SymbolicChain-of-Thought Distillation (SCoTD), a method to train a smaller studentmodel on rationalizations sampled from a significantly larger teacher model.Experiments across several commonsense benchmarks show that: 1) SCoTD enhancesthe performance of the student model in both supervised and few-shot settings,and especially for challenge sets; 2) sampling many reasoning chains perinstance from the teacher is paramount; and 3) after distillation, studentchain-of-thoughts are judged by humans as comparable to the teacher, despiteorders of magnitude fewer parameters. We test several hypotheses regarding whatproperties of chain-of-thought samples are important, e.g., diversity vs.teacher likelihood vs. open-endedness. We release our corpus ofchain-of-thought samples and code.

Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step