English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Scaling neural networks to "large" sizes, with billions of parameters, hasbeen shown to yield impressive results on many challenging problems. However,the inference cost incurred by such large models often prevents theirapplication in most real-world settings. In this paper, we propose a two-stageframework based on distillation that realizes the modelling benefits of thelarge models, while largely preserving the computational benefits of inferencewith more lightweight models. In a nutshell, we use the large teacher models toguide the lightweight student models to only make correct predictions on asubset of "easy" examples; for the "hard" examples, we fall-back to theteacher. Such an approach allows us to efficiently employ large models inpractical scenarios where easy examples are much more frequent than rare hardexamples. Our proposed use of distillation to only handle easy instances allowsfor a more aggressive trade-off in the student size, thereby reducing theamortized cost of inference and achieving better accuracy than standarddistillation. Empirically, we demonstrate the benefits of our approach on bothimage classification and natural language processing benchmarks.

When in Doubt, Summon the Titans: Efficient Inference with Large Models