English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent work has shown that optimizing the Learning Rate (LR) schedule can bea very accurate and efficient way to train deep neural networks. We observethat the rate of change (ROC) of LR has correlation with the training process,but how to use this relationship to control the training to achieve the purposeof improving accuracy? We propose a new method, k-decay, just add an extra itemto the commonly used and easy LR schedule(exp, cosine and polynomial), iseffectively improves the performance of these schedule, also better than thestate-of-the-art algorithms of LR shcedule such as SGDR, CLR and AutoLRS. Inthe k-decay, by adjusting the hyper-parameter \(k\), to generate different LRschedule, when k increases, the performance is improved. We evaluate thek-decay method on CIFAR And ImageNet datasets with different neural networks(ResNet, Wide ResNet). Our experiments show that this method can improve onmost of them. The accuracy has been improved by 1.08\% on the CIFAR-10 datasetand by 2.07 \% on the CIFAR-100 dataset. On the ImageNet, accuracy is improvedby 1.25\%. Our method is not only a general method to be applied other LRShcedule, but also has no additional computational cost.

kDecay: Just adding k-decay items on Learning-Rate Schedule to improve Neural Networks