English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Overfitting negatively impacts the generalization ability of deep neuralnetworks (DNNs) in both natural and adversarial training. Existing methodsstruggle to consistently address different types of overfitting, typicallydesigning strategies that focus separately on either natural or adversarialpatterns. In this work, we adopt a unified perspective by solely focusing onnatural patterns to explore different types of overfitting. Specifically, weexamine the memorization effect in DNNs and reveal a shared behaviour termedover-memorization, which impairs their generalization capacity. This behaviourmanifests as DNNs suddenly becoming high-confidence in predicting certaintraining patterns and retaining a persistent memory for them. Furthermore, whenDNNs over-memorize an adversarial pattern, they tend to simultaneously exhibithigh-confidence prediction for the corresponding natural pattern. Thesefindings motivate us to holistically mitigate different types of overfitting byhindering the DNNs from over-memorization natural patterns. To this end, wepropose a general framework, Distraction Over-Memorization (DOM), whichexplicitly prevents over-memorization by either removing or augmenting thehigh-confidence natural patterns. Extensive experiments demonstrate theeffectiveness of our proposed method in mitigating overfitting across varioustraining paradigms.

On the Over-Memorization During Natural, Robust and Catastrophic Overfitting