English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While pretrained models such as BERT have shown large gains across naturallanguage understanding tasks, their performance can be improved by furthertraining the model on a data-rich intermediate task, before fine-tuning it on atarget task. However, it is still poorly understood when and whyintermediate-task training is beneficial for a given target task. Toinvestigate this, we perform a large-scale study on the pretrained RoBERTamodel with 110 intermediate-target task combinations. We further evaluate alltrained models with 25 probing tasks meant to reveal the specific skills thatdrive transfer. We observe that intermediate tasks requiring high-levelinference and reasoning abilities tend to work best. We also observe thattarget task performance is strongly correlated with higher-level abilities suchas coreference resolution. However, we fail to observe more granularcorrelations between probing and target task performance, highlighting the needfor further work on broad-coverage probing benchmarks. We also observe evidencethat the forgetting of knowledge learned during pretraining may limit ouranalysis, highlighting the need for further work on transfer learning methodsin these settings.

Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?