English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In recent years novel activation functions have been proposed to improve theperformance of neural networks, and they show superior performance compared tothe ReLU counterpart. However, there are environments, where the availabilityof complex activations is limited, and usually only the ReLU is supported. Inthis paper we propose methods that can be used to improve the performance ofReLU networks by using these efficient novel activations during model training.More specifically, we propose ensemble activations that are composed of theReLU and one of these novel activations. Furthermore, the coefficients of theensemble are neither fixed nor learned, but are progressively updated duringthe training process in a way that by the end of the training only the ReLUactivations remain active in the network and the other activations can beremoved. This means that in inference time the network contains ReLUactivations only. We perform extensive evaluations on the ImageNetclassification task using various compact network architectures and variousnovel activation functions. Results show 0.2-0.8% top-1 accuracy gain, whichconfirms the applicability of the proposed methods. Furthermore, we demonstratethe proposed methods on semantic segmentation and we boost the performance of acompact segmentation network by 0.34% mIOU on the Cityscapes dataset.

PEA: Improving the Performance of ReLU Networks for Free by Using Progressive Ensemble Activations