English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper provides a theoretical justification of the superiorclassification performance of deep rectifier networks over shallow rectifiernetworks from the geometrical perspective of piecewise linear (PWL) classifierboundaries. We show that, for a given threshold on the approximation error, therequired number of boundary facets to approximate a general smooth boundarygrows exponentially with the dimension of the data, and thus the number ofboundary facets, referred to as boundary resolution, of a PWL classifier is animportant quality measure that can be used to estimate a lower bound on theclassification errors. However, learning naively an exponentially large numberof boundary facets requires the determination of an exponentially large numberof parameters and also requires an exponentially large number of trainingpatterns. To overcome this issue of "curse of dimensionality", compressiverepresentations of high resolution classifier boundaries are required. To showthe superior compressive power of deep rectifier networks over shallowrectifier networks, we prove that the maximum boundary resolution of a singlehidden layer rectifier network classifier grows exponentially with the numberof units when this number is smaller than the dimension of the patterns. Whenthe number of units is larger than the dimension of the patterns, the growthrate is reduced to a polynomial order. Consequently, the capacity of generatinga high resolution boundary will increase if the same large number of units arearranged in multiple layers instead of a single hidden layer. Taking highdimensional spherical boundaries as examples, we show how deep rectifiernetworks can utilize geometric symmetries to approximate a boundary with thesame accuracy but with a significantly fewer number of parameters than singlehidden layer nets.

On the Compressive Power of Deep Rectifier Networks for High Resolution Representation of Class Boundaries