English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The nonlinearity of activation functions used in deep learning models arecrucial for the success of predictive models. There are several commonly usedsimple nonlinear functions, including Rectified Linear Unit (ReLU) andLeaky-ReLU (L-ReLU). In practice, these functions remarkably enhance the modelaccuracy. However, there is limited insight into the functionality of thesenonlinear activation functions in terms of why certain models perform betterthan others. Here, we investigate the model performance when using ReLU orL-ReLU as activation functions in different model architectures and datadomains. Interestingly, we found that the application of L-ReLU is mostlyeffective when the number of trainable parameters in a model is relativelysmall. Furthermore, we found that the image classification models seem toperform well with L-ReLU in fully connected layers, especially when pre-trainedmodels such as the VGG-16 are used for the transfer learning.

Effects of the Nonlinearity in Activation Functions on the Performance of Deep Learning Models