English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Multi-Task Learning (MTL) networks have emerged as a promising method fortransferring learned knowledge across different tasks. However, MTL must dealwith challenges such as: overfitting to low resource tasks, catastrophicforgetting, and negative task transfer, or learning interference. Often, inNatural Language Processing (NLP), a separate model per task is needed toobtain the best performance. However, many fine-tuning approaches are bothparameter inefficient, i.e., potentially involving one new model per task, andhighly susceptible to losing knowledge acquired during pretraining. We proposea novel Transformer architecture consisting of a new conditional attentionmechanism as well as a set of task-conditioned modules that facilitate weightsharing. Through this construction (a hypernetwork adapter), we achieve moreefficient parameter sharing and mitigate forgetting by keeping half of theweights of a pretrained model fixed. We also use a new multi-task data samplingstrategy to mitigate the negative effects of data imbalance across tasks. Usingthis approach, we are able to surpass single task fine-tuning methods whilebeing parameter and data efficient (using around 66% of the data for weightupdates). Compared to other BERT Large methods on GLUE, our 8-task modelsurpasses other Adapter methods by 2.8% and our 24-task model outperforms by0.7-1.0% models that use MTL and single task fine-tuning. We show that a largervariant of our single multi-task model approach performs competitively across26 NLP tasks and yields state-of-the-art results on a number of test anddevelopment sets. Our code is publicly available athttps://github.com/CAMTL/CA-MTL.

Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data