English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

As an emerging type of Neural Networks (NNs), Transformers are used in manydomains ranging from Natural Language Processing to Autonomous Driving. In thispaper, we study the robustness problem of Transformers, a key characteristic aslow robustness may cause safety concerns. Specifically, we focus onSparsemax-based Transformers and reduce the finding of their maximum robustnessto a Mixed Integer Quadratically Constrained Programming (MIQCP) problem. Wealso design two pre-processing heuristics that can be embedded in the MIQCPencoding and substantially accelerate its solving. We then conduct experimentsusing the application of Land Departure Warning to compare the robustness ofSparsemax-based Transformers against that of the more conventionalMulti-Layer-Perceptron (MLP) NNs. To our surprise, Transformers are notnecessarily more robust, leading to profound considerations in selectingappropriate NN architectures for safety-critical domain applications.

Are Transformers More Robust? Towards Exact Robustness Verification for Transformers