English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In most settings of practical concern, machine learning practitioners know inadvance what end-task they wish to boost with auxiliary tasks. However, widelyused methods for leveraging auxiliary data like pre-training and itscontinued-pretraining variant are end-task agnostic: they rarely, if ever,exploit knowledge of the target task. We study replacing end-task agnosticcontinued training of pre-trained language models with end-task aware trainingof said models. We argue that for sufficiently important end-tasks, thebenefits of leveraging auxiliary data in a task-aware fashion can justifyforgoing the traditional approach of obtaining generic, end-task agnosticrepresentations as with (continued) pre-training. On three differentlow-resource NLP tasks from two domains, we demonstrate that multi-tasking theend-task and auxiliary objectives results in significantly better downstreamtask performance than the widely-used task-agnostic continued pre-trainingparadigm of Gururangan et al. (2020). We next introduce an online meta-learningalgorithm that learns a set of multi-task weights to better balance among ourmultiple auxiliary objectives, achieving further improvements on end-taskperformance and data efficiency.

Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative