English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

A modular design encourages neural models to disentangle and recombinedifferent facets of knowledge to generalise more systematically to new tasks.In this work, we assume that each task is associated with a subset of latentdiscrete skills from a (potentially small) inventory. In turn, skillscorrespond to parameter-efficient (sparse / low-rank) model parameterisations.By jointly learning these and a task-skill allocation matrix, the network foreach task is instantiated as the average of the parameters of active skills. Tofavour non-trivial soft partitions of skills across tasks, we experiment with aseries of inductive biases, such as an Indian Buffet Process prior and atwo-speed learning rate. We evaluate our latent-skill model on two mainsettings: 1) multitask reinforcement learning for grounded instructionfollowing on 8 levels of the BabyAI platform; and 2) few-shot adaptation ofpre-trained text-to-text generative models on CrossFit, a benchmark comprising160 NLP tasks. We find that the modular design of a network significantlyincreases sample efficiency in reinforcement learning and few-shotgeneralisation in supervised learning, compared to baselines with fully shared,task-specific, or conditionally generated parameters where knowledge isentangled across tasks. In addition, we show how discrete skills helpinterpretability, as they yield an explicit hierarchy of tasks.

Combining Modular Skills in Multitask Learning