English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Language models (LMs) with less than 100B parameters are known to performpoorly on chain-of-thought (CoT) reasoning in contrast to large LMs whensolving unseen tasks. In this work, we aim to equip smaller LMs with thestep-by-step reasoning capability by instruction tuning with CoT rationales. Inorder to achieve this goal, we first introduce a new instruction-tuning datasetcalled the CoT Collection, which augments the existing Flan Collection(including only 9 CoT tasks) with additional 1.84 million rationales across1,060 tasks. We show that CoT fine-tuning Flan-T5 (3B & 11B) with CoTCollection enables smaller LMs to have better CoT capabilities on unseen tasks.On the BIG-Bench-Hard (BBH) benchmark, we report an average improvement of+4.34% (Flan-T5 3B) and +2.60% (Flan-T5 11B), in terms of zero-shot taskaccuracy. Furthermore, we show that instruction tuning with CoT Collectionallows LMs to possess stronger few-shot learning capabilities on 4domain-specific tasks, resulting in an improvement of +2.24% (Flan-T5 3B) and+2.37% (Flan-T5 11B), even outperforming ChatGPT utilizing demonstrations untilthe max length by a +13.98% margin. Our code, the CoT Collection data, andmodel checkpoints are publicly available.

The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning