English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Time series forecasting is an important problem across many domains,including predictions of solar plant energy output, electricity consumption,and traffic jam situation. In this paper, we propose to tackle such forecastingproblem with Transformer [1]. Although impressed by its performance in ourpreliminary study, we found its two major weaknesses: (1) locality-agnostics:the point-wise dot-product self-attention in canonical Transformer architectureis insensitive to local context, which can make the model prone to anomalies intime series; (2) memory bottleneck: space complexity of canonical Transformergrows quadratically with sequence length $L$, making directly modeling longtime series infeasible. In order to solve these two issues, we first proposeconvolutional self-attention by producing queries and keys with causalconvolution so that local context can be better incorporated into attentionmechanism. Then, we propose LogSparse Transformer with only $O(L(\log L)^{2})$memory cost, improving forecasting accuracy for time series with finegranularity and strong long-term dependencies under constrained memory budget.Our experiments on both synthetic data and real-world datasets show that itcompares favorably to the state-of-the-art.

Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting