English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We present a new method LiST is short for Lite Prompted Self-Training forparameter-efficient fine-tuning of large pre-trained language models (PLMs) forfew-shot learning. LiST improves over recent methods that adopt prompt-basedfine-tuning (FN) using two key techniques. The first is the use ofself-training to leverage large amounts of unlabeled data for prompt-based FNin few-shot settings. We use self-training in conjunction with meta-learningfor re-weighting noisy pseudo-prompt labels. Self-training is expensive as itrequires updating all the model parameters repetitively. Therefore, we use asecond technique for light-weight fine-tuning where we introduce a small numberof task-specific parameters that are fine-tuned during self-training whilekeeping the PLM encoder frozen. Our experiments show that LiST can effectivelyleverage unlabeled data to improve the model performance for few-shot learning.Additionally, the fine-tuning is efficient as it only updates a smallpercentage of parameters and the overall model footprint is reduced sinceseveral tasks can share a common PLM encoder as backbone. A comprehensive studyon six NLU tasks demonstrate LiST to improve by 35% over classic fine-tuningand 6% over prompt-based FN with 96% reduction in number of trainableparameters when fine-tuned with no more than 30 labeled examples from eachtask. With only 14M tunable parameters, LiST outperforms GPT-3 in-contextlearning by 33% on few-shot NLU tasks.

LiST: Lite Prompted Self-training Makes Parameter-Efficient Few-shot Learners