English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep learning methods achieve great success recently on many computer visionproblems, with image classification and object detection as the prominentexamples. In spite of these practical successes, optimization of deep networksremains an active topic in deep learning research. In this work, we focus oninvestigation of the network solution properties that can potentially lead togood performance. Our research is inspired by theoretical and empirical resultsthat use orthogonal matrices to initialize networks, but we are interested ininvestigating how orthogonal weight matrices perform when network trainingconverges. To this end, we propose to constrain the solutions of weightmatrices in the orthogonal feasible set during the whole process of networktraining, and achieve this by a simple yet effective method called SingularValue Bounding (SVB). In SVB, all singular values of each weight matrix aresimply bounded in a narrow band around the value of 1. Based on the samemotivation, we also propose Bounded Batch Normalization (BBN), which improvesBatch Normalization by removing its potential risk of ill-conditioned layertransform. We present both theoretical and empirical results to justify ourproposed methods. Experiments on benchmark image classification datasets showthe efficacy of our proposed SVB and BBN. In particular, we achieve thestate-of-the-art results of 3.06% error rate on CIFAR10 and 16.90% on CIFAR100,using off-the-shelf network architectures (Wide ResNets). Our preliminaryresults on ImageNet also show the promise in large-scale learning.

Improving training of deep neural networks via Singular Value Bounding