English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Parameter-efficient tuning aims to distill knowledge for downstream tasks byoptimizing a few introduced parameters while freezing the pretrained languagemodels (PLMs). Continuous prompt tuning which prepends a few trainable vectorsto the embeddings of input is one of these methods and has drawn much attentiondue to its effectiveness and efficiency. This family of methods can beillustrated as exerting nonlinear transformations of hidden states inside PLMs.However, a natural question is ignored: can the hidden states be directly usedfor classification without changing them? In this paper, we aim to answer thisquestion by proposing a simple tuning method which only introduces threetrainable vectors. Firstly, we integrate all layers hidden states using theintroduced vectors. And then, we input the integrated hidden state(s) to atask-specific linear classifier to predict categories. This scheme is similarto the way ELMo utilises hidden states except that they feed the hidden statesto LSTM-based models. Although our proposed tuning scheme is simple, itachieves comparable performance with prompt tuning methods like P-tuning andP-tuning v2, verifying that original hidden states do contain usefulinformation for classification tasks. Moreover, our method has an advantageover prompt tuning in terms of time and the number of parameters.

Parameter-Efficient Tuning by Manipulating Hidden States of Pretrained Language Models For Classification Tasks