English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large language models (LLMs) such as T0, FLAN, and OPT-IML, excel inmulti-tasking under a unified instruction-following paradigm, where they alsoexhibit remarkable generalization abilities to unseen tasks. Despite theirimpressive performance, these LLMs, with sizes ranging from several billion tohundreds of billions of parameters, demand substantial computational resources,making their training and inference expensive and inefficient. Furthermore,adapting these models to downstream applications, particularly complex tasks,is often unfeasible due to the extensive hardware requirements for finetuning,even when utilizing parameter-efficient approaches such as prompt tuning.Additionally, the most powerful multi-task LLMs, such as OPT-IML-175B andFLAN-PaLM-540B, are not publicly accessible, severely limiting theircustomization potential. To address these challenges, we introduce a pretrainedsmall scorer, Cappy, designed to enhance the performance and efficiency ofmulti-task LLMs. With merely 360 million parameters, Cappy functions eitherindependently on classification tasks or serve as an auxiliary component forLLMs, boosting their performance. Moreover, Cappy enables efficientlyintegrating downstream supervision without requiring LLM finetuning nor theaccess to their parameters. Our experiments demonstrate that, when workingindependently on 11 language understanding tasks from PromptSource, Cappyoutperforms LLMs that are several orders of magnitude larger. Besides, on 45complex tasks from BIG-Bench, Cappy boosts the performance of the advancedmulti-task LLM, FLAN-T5, by a large margin. Furthermore, Cappy is flexible tocooperate with other LLM adaptations, including finetuning and in-contextlearning, offering additional performance enhancement.

Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer