English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transformer models have shown great success in natural language processing;however, their potential remains mostly unexplored for dynamical systems. Inthis work, we investigate the optimal output estimation problem usingtransformers, which generate output predictions using all the past ones.Particularly, we train the transformer using various distinct systems and thenevaluate the performance on unseen systems with unknown dynamics. Empirically,the trained transformer adapts exceedingly well to different unseen systems andeven matches the optimal performance given by the Kalman filter for linearsystems. In more complex settings with non-i.i.d. noise, time-varying dynamics,and nonlinear dynamics like a quadrotor system with unknown parameters,transformers also demonstrate promising results. To support our experimentalfindings, we provide statistical guarantees that quantify the amount oftraining data required for the transformer to achieve a desired excess risk.Finally, we point out some limitations by identifying two classes of problemsthat lead to degraded performance, highlighting the need for caution when usingtransformers for control and estimation.

Can Transformers Learn Optimal Filtering for Unknown Systems?