English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Multi-task learning (MTL) models have demonstrated impressive results incomputer vision, natural language processing, and recommender systems. Eventhough many approaches have been proposed, how well these approaches balancedifferent tasks on each parameter still remains unclear. In this paper, wepropose to measure the task dominance degree of a parameter by the totalupdates of each task on this parameter. Specifically, we compute the totalupdates by the exponentially decaying Average of the squared Updates (AU) on aparameter from the corresponding task.Based on this novel metric, we observethat many parameters in existing MTL methods, especially those in the highershared layers, are still dominated by one or several tasks. The dominance of AUis mainly due to the dominance of accumulative gradients from one or severaltasks. Motivated by this, we propose a Task-wise Adaptive learning rateapproach, AdaTask in short, to separate the \emph{accumulative gradients} andhence the learning rate of each task for each parameter in adaptive learningrate approaches (e.g., AdaGrad, RMSProp, and Adam). Comprehensive experimentson computer vision and recommender system MTL datasets demonstrate that AdaTasksignificantly improves the performance of dominated tasks, resulting SOTAaverage task-wise performance. Analysis on both synthetic and real-worlddatasets shows AdaTask balance parameters in every shared layer well.

AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning