English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Delta tuning (DET, also known as parameter-efficient tuning) is deemed as thenew paradigm for using pre-trained language models (PLMs). Up to now, variousDETs with distinct design elements have been proposed, achieving performance onpar with fine-tuning. However, the mechanisms behind the above success arestill under-explored, especially the connections among various DETs. To fathomthe mystery, we hypothesize that the adaptations of different DETs could all bereparameterized as low-dimensional optimizations in a unified optimizationsubspace, which could be found by jointly decomposing independent solutions ofdifferent DETs. Then we explore the connections among different DETs byconducting optimization within the subspace. In experiments, we find that, fora certain DET, conducting optimization simply in the subspace could achievecomparable performance to its original space, and the found solution in thesubspace could be transferred to another DET and achieve non-trivialperformance. We also visualize the performance landscape of the subspace andfind that there exists a substantial region where different DETs all performwell. Finally, we extend our analysis and show the strong connections betweenfine-tuning and DETs.

Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning