English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Few-shot learning is valuable in many real-world applications, but learning ageneralizable model without overfitting to the few labeled datapoints ischallenging. In this work, we focus on Few-shot Learning with Auxiliary Data(FLAD), a training paradigm that assumes access to auxiliary data duringfew-shot learning in hopes of improving generalization. Previous works haveproposed automated methods for mixing auxiliary and target data, but thesemethods typically scale linearly (or worse) with the number of auxiliarydatasets, limiting their practicality. In this work we relate FLAD to theexplore-exploit dilemma that is central to the multi-armed bandit setting andderive algorithms whose computational complexity is independent of the numberof auxiliary datasets, allowing us to scale to 100x more auxiliary datasetsthan prior methods. We propose two algorithms -- EXP3-FLAD and UCB1-FLAD -- andcompare them with prior FLAD methods that either explore or exploit, findingthat the combination of exploration and exploitation is crucial. Throughextensive experimentation we find that our methods outperform all pre-existingFLAD methods by 4% and lead to the first 3 billion parameter language modelsthat outperform the 175 billion parameter GPT-3. Overall, our work suggeststhat the discovery of better, more efficient mixing strategies for FLAD mayprovide a viable path towards substantially improving generalization infew-shot learning.

Improving Few-Shot Generalization by Exploring and Exploiting Auxiliary Data