English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Transfer learning has fundamentally changed the landscape of natural languageprocessing (NLP) research. Many existing state-of-the-art models are firstpre-trained on a large text corpus and then fine-tuned on downstream tasks.However, due to limited data resources from downstream tasks and the extremelylarge capacity of pre-trained models, aggressive fine-tuning often causes theadapted model to overfit the data of downstream tasks and forget the knowledgeof the pre-trained model. To address the above issue in a more principledmanner, we propose a new computational framework for robust and efficientfine-tuning for pre-trained language models. Specifically, our proposedframework contains two important ingredients: 1. Smoothness-inducingregularization, which effectively manages the capacity of the model; 2. Bregmanproximal point optimization, which is a class of trust-region methods and canprevent knowledge forgetting. Our experiments demonstrate that our proposedmethod achieves the state-of-the-art performance on multiple NLP benchmarks.

SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization