English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Adversarial training (AT) is a widely recognized defense mechanism to gainthe robustness of deep neural networks against adversarial attacks. It is builton min-max optimization (MMO), where the minimizer (i.e., defender) seeks arobust model to minimize the worst-case training loss in the presence ofadversarial examples crafted by the maximizer (i.e., attacker). However, theconventional MMO method makes AT hard to scale. Thus, Fast-AT (Wong et al.,2020) and other recent algorithms attempt to simplify MMO by replacing itsmaximization step with the single gradient sign-based attack generation step.Although easy to implement, Fast-AT lacks theoretical guarantees, and itsempirical performance is unsatisfactory due to the issue of robust catastrophicoverfitting when training with strong adversaries. In this paper, we advanceFast-AT from the fresh perspective of bi-level optimization (BLO). We firstshow that the commonly-used Fast-AT is equivalent to using a stochasticgradient algorithm to solve a linearized BLO problem involving a signoperation. However, the discrete nature of the sign operation makes itdifficult to understand the algorithm performance. Inspired by BLO, we designand analyze a new set of robust training algorithms termed Fast Bi-level AT(Fast-BAT), which effectively defends sign-based projected gradient descent(PGD) attacks without using any gradient sign method or explicit robustregularization. In practice, we show our method yields substantial robustnessimprovements over baselines across multiple models and datasets. Codes areavailable at https://github.com/OPTML-Group/Fast-BAT.

Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization