English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Adversarial training has been shown to be one of the most effectiveapproaches to improve the robustness of deep neural networks. It is formalizedas a min-max optimization over model weights and adversarial perturbations,where the weights can be optimized through gradient descent methods like SGD.In this paper, we show that treating model weights as random variables allowsfor enhancing adversarial training through \textbf{S}econd-Order\textbf{S}tatistics \textbf{O}ptimization (S$^2$O) with respect to the weights.By relaxing a common (but unrealistic) assumption of previous PAC-Bayesianframeworks that all weights are statistically independent, we derive animproved PAC-Bayesian adversarial generalization bound, which suggests thatoptimizing second-order statistics of weights can effectively tighten thebound. In addition to this theoretical insight, we conduct an extensive set ofexperiments, which show that S$^2$O not only improves the robustness andgeneralization of the trained neural networks when used in isolation, but alsointegrates easily in state-of-the-art adversarial training techniques likeTRADES, AWP, MART, and AVMixup, leading to a measurable improvement of thesetechniques. The code is available at \url{https://github.com/Alexkael/S2O}.

Enhancing Adversarial Training with Second-Order Statistics of Weights