English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-trained language models (PLMs) have achieved remarkable results on NLPtasks but at the expense of huge parameter sizes and the consequentcomputational costs. In this paper, we propose Variator, a parameter-efficientacceleration method that enhances computational efficiency throughplug-and-play compression plugins. Compression plugins are designed to reducethe sequence length via compressing multiple hidden vectors into one andtrained with original PLMs frozen. Different from traditional modelacceleration methods, which compress PLMs to smaller sizes, Variator offers twodistinct advantages: (1) In real-world applications, the plug-and-play natureof our compression plugins enables dynamic selection of different compressionplugins with varying acceleration ratios based on the current workload. (2) Thecompression plugin comprises a few compact neural network layers with minimalparameters, significantly saving storage and memory overhead, particularly inscenarios with a growing number of tasks. We validate the effectiveness ofVariator on seven datasets. Experimental results show that Variator can save53% computational costs using only 0.9% additional parameters with aperformance drop of less than 2%. Moreover, when the model scales to billionsof parameters, Variator matches the strong performance of uncompressed PLMs.

Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules