English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The increasing adoption of natural language processing (NLP) models acrossindustries has led to practitioners' need for machine learning systems tohandle these models efficiently, from training to serving them in production.However, training, deploying, and updating multiple models can be complex,costly, and time-consuming, mainly when using transformer-based pre-trainedlanguage models. Multi-Task Learning (MTL) has emerged as a promising approachto improve efficiency and performance through joint training, rather thantraining separate models. Motivated by this, we first provide an overview oftransformer-based MTL approaches in NLP. Then, we discuss the challenges andopportunities of using MTL approaches throughout typical ML lifecycle phases,specifically focusing on the challenges related to data engineering, modeldevelopment, deployment, and monitoring phases. This survey focuses ontransformer-based MTL architectures and, to the best of our knowledge, is novelin that it systematically analyses how transformer-based MTL in NLP fits intoML lifecycle phases. Furthermore, we motivate research on the connectionbetween MTL and continual learning (CL), as this area remains unexplored. Webelieve it would be practical to have a model that can handle both MTL and CL,as this would make it easier to periodically re-train the model, update it dueto distribution shifts, and add new capabilities to meet real-worldrequirements.

Challenges and Opportunities of Using Transformer-Based Multi-Task Learning in NLP Through ML Lifecycle: A Survey