English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Representational sparsity is known to affect robustness to inputperturbations in deep neural networks (DNNs), but less is known about how thesemantic content of representations affects robustness. Class selectivity-thevariability of a unit's responses across data classes or dimensions-is one wayof quantifying the sparsity of semantic representations. Given recent evidencethat class selectivity may not be necessary for, and in some cases can impairgeneralization, we investigate whether it also confers robustness (orvulnerability) to perturbations of input data. We found that networksregularized to have lower levels of class selectivity were more robust toaverage-case (naturalistic) perturbations, while networks with higher classselectivity are more vulnerable. In contrast, class selectivity increasesrobustness to multiple types of worst-case (i.e. white box adversarial)perturbations, suggesting that while decreasing class selectivity is helpfulfor average-case perturbations, it is harmful for worst-case perturbations. Toexplain this difference, we studied the dimensionality of the networks'representations: we found that the dimensionality of early-layerrepresentations is inversely proportional to a network's class selectivity, andthat adversarial samples cause a larger increase in early-layer dimensionalitythan corrupted samples. Furthermore, the input-unit gradient is more variableacross samples and units in high-selectivity networks compared tolow-selectivity networks. These results lead to the conclusion that unitsparticipate more consistently in low-selectivity regimes compared tohigh-selectivity regimes, effectively creating a larger attack surface andhence vulnerability to worst-case perturbations.

Linking average- and worst-case perturbation robustness via class selectivity and dimensionality