English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent development of large-scale pre-trained language models (PLM) havesignificantly improved the capability of models in various NLP tasks, in termsof performance after task-specific fine-tuning and zero-shot / few-shotlearning. However, many of such models come with a dauntingly huge size thatfew institutions can afford to pre-train, fine-tune or even deploy, whilemoderate-sized models usually lack strong generalized few-shot learningcapabilities. In this paper, we first elaborate the current obstacles of usingPLM models in terms of the Impossible Triangle: 1) moderate model size, 2)state-of-the-art few-shot learning capability, and 3) state-of-the-artfine-tuning capability. We argue that all existing PLM models lack one or moreproperties from the Impossible Triangle. To remedy these missing properties ofPLMs, various techniques have been proposed, such as knowledge distillation,data augmentation and prompt learning, which inevitably brings additional workto the application of PLMs in real scenarios. We then offer insights intofuture research directions of PLMs to achieve the Impossible Triangle, andbreak down the task into several key phases.

Impossible Triangle: What's Next for Pre-trained Language Models?