English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Self-supervised learning has been actively studied in time series domainrecently, especially for masked reconstruction. Most of these methods followthe "Pre-training + Fine-tuning" paradigm in which a new decoder replaces thepre-trained decoder to fit for a specific downstream task, leading toinconsistency of upstream and downstream tasks. In this paper, we first pointout that the unification of task objectives and adaptation for task difficultyare critical for bridging the gap between time series masked reconstruction andforecasting. By reserving the pre-trained mask token during fine-tuning stage,the forecasting task can be taken as a special case of masked reconstruction,where the future values are masked and reconstructed based on history values.It guarantees the consistency of task objectives but there is still a gap intask difficulty. Because masked reconstruction can utilize contextualinformation while forecasting can only use historical information toreconstruct. To further mitigate the existed gap, we propose a simple yeteffective prompt token tuning (PT-Tuning) paradigm, in which all pre-trainedparameters are frozen and only a few trainable prompt tokens are added toextended mask tokens in element-wise manner. Extensive experiments onreal-world datasets demonstrate the superiority of our proposed paradigm withstate-of-the-art performance compared to representation learning and end-to-endsupervised forecasting methods.

PT-Tuning: Bridging the Gap between Time Series Masked Reconstruction and Forecasting via Prompt Token Tuning