English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transfer learning (TL) in natural language processing (NLP) has seen a surgeof interest in recent years, as pre-trained models have shown an impressiveability to transfer to novel tasks. Three main strategies have emerged formaking use of multiple supervised datasets during fine-tuning: training on anintermediate task before training on the target task (STILTs), using multi-tasklearning (MTL) to train jointly on a supplementary task and the target task(pairwise MTL), or simply using MTL to train jointly on all available datasets(MTL-ALL). In this work, we compare all three TL methods in a comprehensiveanalysis on the GLUE dataset suite. We find that there is a simple heuristicfor when to use one of these techniques over the other: pairwise MTL is betterthan STILTs when the target task has fewer instances than the supporting taskand vice versa. We show that this holds true in more than 92% of applicablecases on the GLUE dataset and validate this hypothesis with experiments varyingdataset size. The simplicity and effectiveness of this heuristic is surprisingand warrants additional exploration by the TL community. Furthermore, we findthat MTL-ALL is worse than the pairwise methods in almost every case. We hopethis study will aid others as they choose between TL methods for NLP tasks.

When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learning