English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Recently, prompt tuning (PT) has gained increasing attention as aparameter-efficient way of tuning pre-trained language models (PLMs). Despiteextensively reducing the number of tunable parameters and achieving satisfyingperformance, PT is training-inefficient due to its slow convergence. To improvePT's training efficiency, we first make some novel observations about theprompt transferability of "partial PLMs", which are defined by compressing aPLM in depth or width. We observe that the soft prompts learned by differentpartial PLMs of various sizes are similar in the parameter space, implying thatthese soft prompts could potentially be transferred among partial PLMs.Inspired by these observations, we propose Fast Prompt Tuning (FPT), whichstarts by conducting PT using a small-scale partial PLM, and then progressivelyexpands its depth and width until the full-model size. After each expansion, werecycle the previously learned soft prompts as initialization for the enlargedpartial PLM and then proceed PT. We demonstrate the feasibility of FPT on 5tasks and show that FPT could save over 30% training computations whileachieving comparable performance.

FPT: Improving Prompt Tuning Efficiency via Progressive Training