English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

In this paper we propose and investigate a novel nonlinear unit, called $L_p$unit, for deep neural networks. The proposed $L_p$ unit receives signals fromseveral projections of a subset of units in the layer below and computes anormalized $L_p$ norm. We notice two interesting interpretations of the $L_p$unit. First, the proposed unit can be understood as a generalization of anumber of conventional pooling operators such as average, root-mean-square andmax pooling widely used in, for instance, convolutional neural networks (CNN),HMAX models and neocognitrons. Furthermore, the $L_p$ unit is, to a certaindegree, similar to the recently proposed maxout unit (Goodfellow et al., 2013)which achieved the state-of-the-art object recognition results on a number ofbenchmark datasets. Secondly, we provide a geometrical interpretation of theactivation function based on which we argue that the $L_p$ unit is moreefficient at representing complex, nonlinear separating boundaries. Each $L_p$unit defines a superelliptic boundary, with its exact shape defined by theorder $p$. We claim that this makes it possible to model arbitrarily shaped,curved boundaries more efficiently by combining a few $L_p$ units of differentorders. This insight justifies the need for learning different orders for eachunit in the model. We empirically evaluate the proposed $L_p$ units on a numberof datasets and show that multilayer perceptrons (MLP) consisting of the $L_p$units achieve the state-of-the-art results on a number of benchmark datasets.Furthermore, we evaluate the proposed $L_p$ unit on the recently proposed deeprecurrent neural networks (RNN).

Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks