English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Self-attention mechanisms are commonly included in a convolutional neuralnetworks to achieve an improved efficiency performance balance. However, addingself-attention mechanisms adds additional hyperparameters to tune for theapplication at hand. In this work we propose a novel type of DNN analysiscalled Multi-Scale Class Representational Response Similarity Analysis(ClassRepSim) which can be used to identify specific design interventions thatlead to more efficient self-attention convolutional neural networkarchitectures. Using insights grained from ClassRepSim we propose the SpatialTransformed Attention Condenser (STAC) module, a novel attention-condenserbased self-attention module. We show that adding STAC modules to ResNet stylearchitectures can result in up to a 1.6% increase in top-1 accuracy compared tovanilla ResNet models and up to a 0.5% increase in top-1 accuracy compared toSENet models on the ImageNet64x64 dataset, at the cost of up to 1.7% increasein FLOPs and 2x the number of parameters. In addition, we demonstrate thatresults from ClassRepSim analysis can be used to select an effectiveparameterization of the STAC module resulting in competitive performancecompared to an extensive parameter search.

Systematic Architectural Design of Scale Transformed Attention Condenser DNNs via Multi-Scale Class Representational Response Similarity Analysis