English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deploying large language models (LLMs) is challenging because they are memoryinefficient and compute-intensive for practical applications. In reaction,researchers train smaller task-specific models by either finetuning with humanlabels or distilling using LLM-generated labels. However, finetuning anddistillation require large amounts of training data to achieve comparableperformance to LLMs. We introduce Distilling step-by-step, a new mechanism that(a) trains smaller models that outperform LLMs, and (b) achieves so byleveraging less training data needed by finetuning or distillation. Our methodextracts LLM rationales as additional supervision for training small modelswithin a multi-task framework. We present three findings across 4 NLPbenchmarks: First, compared to both finetuning and distillation, our mechanismachieves better performance with much fewer labeled/unlabeled trainingexamples. Second, compared to few-shot prompted LLMs, we achieve betterperformance using substantially smaller model sizes. Third, we reduce both themodel size and the amount of data required to outperform LLMs; our finetuned770M T5 model outperforms the few-shot prompted 540B PaLM model using only 80%of available data on a benchmark, whereas standard finetuning the same T5 modelstruggles to match even by using 100% of the dataset. We release the code at:https://github.com/google-research/distilling-step-by-step .

Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes