English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With the rapid development of Deep Neural Networks (DNNs), various networkmodels that show strong computing power and impressive expressive power areproposed. However, there is no comprehensive informational interpretation ofDNNs from the perspective of information theory. Due to the nonlinear functionand the uncertain number of layers and neural units used in the DNNs, thenetwork structure shows nonlinearity and complexity. With the typical DNNsnamed Convolutional Arithmetic Circuits (ConvACs), the complex DNNs can beconverted into mathematical formula. Thus, we can use rigorous mathematicaltheory especially the information theory to analyse the complicated DNNs. Inthis paper, we propose a novel information scaling law scheme that caninterpret the network's inner organization by information theory. First, weshow the informational interpretation of the activation function. Secondly, weprove that the information entropy increases when the information istransmitted through the ConvACs. Finally, we propose the information scalinglaw of ConvACs through making a reasonable assumption.

Information Scaling Law of Deep Neural Networks