English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Although large language models have achieved impressive zero-shot ability,the huge model size generally incurs high cost. Recently, semi-parametriclanguage models, which augment a smaller language model with an externalretriever, have demonstrated promising language modeling capabilities. However,it remains unclear whether such semi-parametric language models can performcompetitively well as their fully-parametric counterparts on zero-shotgeneralization to downstream tasks. In this work, we introduce $\text{Zemi}$, azero-shot semi-parametric language model. To our best knowledge, this is thefirst semi-parametric language model that can demonstrate strong zero-shotperformance on a wide range of held-out unseen tasks. We train $\text{Zemi}$with a novel semi-parametric multitask prompted training paradigm, which showssignificant improvement compared with the parametric multitask training asproposed by T0. Specifically, we augment the multitask training and zero-shotevaluation with retrieval from a large-scale task-agnostic unlabeled corpus. Inorder to incorporate multiple potentially noisy retrieved augmentations, wefurther propose a novel $\text{augmentation fusion}$ module leveragingperceiver resampler and gated cross-attention. Notably, our proposed$\text{Zemi}_\text{LARGE}$ outperforms T0-3B by 16% on all seven evaluationtasks while being 3.9x smaller in model size.

Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks