English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Deep neural network with rectified linear units (ReLU) is getting more andmore popular recently. However, the derivatives of the function represented bya ReLU network are not continuous, which limit the usage of ReLU network tosituations only when smoothness is not required. In this paper, we constructdeep neural networks with rectified power units (RePU), which can give betterapproximations for smooth functions. Optimal algorithms are proposed toexplicitly build neural networks with sparsely connected RePUs, which we callPowerNets, to represent polynomials with no approximation error. For generalsmooth functions, we first project the function to their polynomialapproximations, then use the proposed algorithms to construct correspondingPowerNets. Thus, the error of best polynomial approximation provides an upperbound of the best RePU network approximation error. For smooth functions inhigher dimensional Sobolev spaces, we use fast spectral transforms fortensor-product grid and sparse grid discretization to get polynomialapproximations. Our constructive algorithms show clearly a close connectionbetween spectral methods and deep neural networks: a PowerNet with $n$ layerscan exactly represent polynomials up to degree $s^n$, where $s$ is the power ofRePUs. The proposed PowerNets have potential applications in the situationswhere high-accuracy is desired or smoothness is required.

PowerNet: Efficient Representations of Polynomials and Smooth Functions by Deep Neural Networks with Rectified Power Units