English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

High quality explanations of neural networks (NNs) should exhibit two keyproperties. Completeness ensures that they accurately reflect a network'sfunction and interpretability makes them understandable to humans. Manyexisting methods provide explanations of individual neurons within a network.In this work we provide evidence that for AlexNet pretrained on ImageNet,neuron-based explanation methods sacrifice both completeness andinterpretability compared to activation principal components. Neurons are apoor basis for AlexNet embeddings because they don't account for thedistributed nature of these representations. By examining two quantitativemeasures of completeness and conducting a user study to measureinterpretability, we show the most important principal components provide morecomplete and interpretable explanations than the most important neurons. Muchof the activation variance may be explained by examining relatively fewhigh-variance PCs, as opposed to studying every neuron. These principalcomponents also strongly affect network function, and are significantly moreinterpretable than neurons. Our findings suggest that explanation methods fornetworks like AlexNet should avoid using neurons as a basis for embeddings andinstead choose a basis, such as principal components, which accounts for thehigh dimensional and distributed nature of a network's internalrepresentations. Interactive demo and code available athttps://ndey96.github.io/neuron-explanations-sacrifice.

Neuron-based explanations of neural networks sacrifice completeness and interpretability