Entropy Learning in Neural Network | Zendy

Geok See Ng | Zendy; Daming Shi | Zendy; Abdul Wahab | Zendy; Harminder Singh | Zendy

AI Assistant Blog Pricing

Home ZAIA Blog

Open Access

Entropy Learning in Neural Network

Author(s) -

Geok See Ng,

Daming Shi,

Abdul Wahab,

Harminder Singh

Publication year - 2017

Publication title -

asean journal on science and technology for development

Language(s) - English

Resource type - Journals

eISSN - 2224-9028

pISSN - 0217-5460

DOI - 10.29037/ajstd.362

Subject(s) - artificial neural network , computer science , entropy (arrow of time) , artificial intelligence , machine learning , physics , quantum mechanics

In this paper, entropy term is used in the learning phase of a neural network. As learning progresses, more hidden nodes get into saturation. The early creation of such hidden nodes may impair generalisation. Hence entropy approach is proposed to dampen the early creation of such nodes. The entropy learning also helps to increase the importance of relevant nodes while dampening the less important nodes. At the end of learning, the less important nodes can then be eliminated to reduce the memory requirements of the neural network.

The content you want is available to Zendy users.

Already have an account? Click here to sign in.

Having issues? You can contact us here

Accelerating Research