English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite the remarkable success achieved by deep learning algorithms invarious domains, such as computer vision, they remain vulnerable to adversarialperturbations. Adversarial Training (AT) stands out as one of the mosteffective solutions to address this issue; however, single-step AT can lead toCatastrophic Overfitting (CO). This scenario occurs when the adversariallytrained network suddenly loses robustness against multi-step attacks likeProjected Gradient Descent (PGD). Although several approaches have beenproposed to address this problem in Convolutional Neural Networks (CNNs), wefound out that they do not perform well when applied to Vision Transformers(ViTs). In this paper, we propose Blacksmith, a novel training strategy toovercome the CO problem, specifically in ViTs. Our approach utilizes either ofPGD-2 or Fast Gradient Sign Method (FGSM) randomly in a mini-batch during theadversarial training of the neural network. This will increase the diversity ofour training attacks, which could potentially mitigate the CO issue. To managethe increased training time resulting from this combination, we craft the PGD-2attack based on only the first half of the layers, while FGSM is appliedend-to-end. Through our experiments, we demonstrate that our novel methodeffectively prevents CO, achieves PGD-2 level performance, and outperformsother existing techniques including N-FGSM, which is the state-of-the-artmethod in fast training for CNNs.

Blacksmith: Fast Adversarial Training of Vision Transformers via a Mixture of Single-step and Multi-step Methods