English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Prompt tuning is an emerging way of adapting pre-trained language models todownstream tasks. However, the existing studies are mainly to add prompts tothe input sequence. This way would not work as expected due to the intermediatemulti-head self-attention and feed-forward network computation, making modeloptimization not very smooth. Hence, we propose a novel tuning way called layertuning, aiming to add learnable parameters in Transformer layers. Specifically,we focus on layer tuning for feed-forward network in the Transformer, namelyFL-tuning. It introduces additional units into the hidden layer of eachfeed-forward network. We conduct extensive experiments on the public CLUEbenchmark. The results show that: 1) Our FL-tuning outperforms prompt tuningmethods under both full-data and few-shot settings in almost all cases. Inparticular, it improves accuracy by 17.93% (full-data setting) on WSC 1.0 andF1 by 16.142% (few-shot setting) on CLUENER over P-tuning v2. 2) Our FL-tuningis more stable and converges about 1.17 times faster than P-tuning v2. 3) Withonly about 3% of Transformer's parameters to be trained, FL-tuning iscomparable with fine-tuning on most datasets, and significantly outperformsfine-tuning (e.g., accuracy improved by 12.9% on WSC 1.1) on several datasets.The source codes are available at https://github.com/genggui001/FL-Tuning.

FL-Tuning: Layer Tuning for Feed-Forward Network in Transformer