English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In the era of Deep Neural Network based solutions for a variety of real-lifetasks, having a compact and energy-efficient deployable model has become fairlyimportant. Most of the existing deep architectures use Rectifier Linear Unit(ReLU) activation. In this paper, we propose a novel idea of rotating the ReLUactivation to give one more degree of freedom to the architecture. We show thatthis activation wherein the rotation is learned via training results in theelimination of those parameters/filters in the network which are not importantfor the task. In other words, rotated ReLU seems to be doing implicitsparsification. The slopes of the rotated ReLU activations act as coarsefeature extractors and unnecessary features can be eliminated beforeretraining. Our studies indicate that features always choose to pass through alesser number of filters in architectures such as ResNet and its variants.Hence, by rotating the ReLU, the weights or the filters that are not necessaryare automatically identified and can be dropped thus giving rise to significantsavings in memory and computation. Furthermore, in some cases, we also noticethat along with saving in memory and computation we also obtain improvementover the reported performance of the corresponding baseline work in the populardatasets such as MNIST, CIFAR-10, CIFAR-100, and SVHN.

Rotate the ReLU to implicitly sparsify deep networks