English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Orthogonality regularization has been developed to prevent deep CNNs fromtraining instability and feature redundancy. Among existing proposals, kernelorthogonality regularization enforces orthogonality by minimizing the residualbetween the Gram matrix formed by convolutional filters and the orthogonalitymatrix.  We propose a novel measure for achieving better orthogonality among filters,which disentangles diagonal and correlation information from the residual. Themodel equipped with the measure under the principle of imposing strictorthogonality between filters surpasses previous regularization methods innear-orthogonality. Moreover, we observe the benefits of improved strict filterorthogonality in relatively shallow models, but as model depth increases, theperformance gains in models employing strict kernel orthogonality decreasesharply.  Furthermore, based on the observation of the potential conflict betweenstrict kernel orthogonality and growing model capacity, we propose a relaxationtheory on kernel orthogonality regularization. The relaxed kernel orthogonalityachieves enhanced performance on models with increased capacity, shedding lighton the burden of strict kernel orthogonality on deep model performance.  We conduct extensive experiments with our kernel orthogonality regularizationtoolkit on ResNet and WideResNet in CIFAR-10 and CIFAR-100. We observestate-of-the-art gains in model performance from the toolkit, which includesboth strict orthogonality and relaxed orthogonality regularization, and obtainmore robust models with expressive features. These experiments demonstrate theefficacy of our toolkit and subtly provide insights into the often overlookedchallenges posed by strict orthogonality, addressing the burden of strictorthogonality on capacity-rich models.

Towards Better Orthogonality Regularization with Disentangled Norm in Training Deep CNNs