English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Fine-tuning is widely used as the default algorithm for transfer learningfrom pre-trained models. Parameter inefficiency can however arise when, duringtransfer learning, all the parameters of a large pre-trained model need to beupdated for individual downstream tasks. As the number of parameters grows,fine-tuning is prone to overfitting and catastrophic forgetting. In addition,full fine-tuning can become prohibitively expensive when the model is used formany tasks. To mitigate this issue, parameter-efficient transfer learningalgorithms, such as adapters and prefix tuning, have been proposed as a way tointroduce a few trainable parameters that can be plugged into large pre-trainedlanguage models such as BERT, and HuBERT. In this paper, we introduce theSpeech UndeRstanding Evaluation (SURE) benchmark for parameter-efficientlearning for various speech-processing tasks. Additionally, we introduce a newadapter, ConvAdapter, based on 1D convolution. We show that ConvAdapteroutperforms the standard adapters while showing comparable performance againstprefix tuning and LoRA with only 0.94% of trainable parameters on some of thetask in SURE. We further explore the effectiveness of parameter efficienttransfer learning for speech synthesis task such as Text-to-Speech (TTS).

Evaluating Parameter-Efficient Transfer Learning Approaches on SURE Benchmark for Speech Understanding