English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Efforts to improve the learning abilities of neural networks have focusedmostly on the role of optimization methods rather than on weightinitializations. Recent findings, however, suggest that neural networks rely onlucky random initial weights of subnetworks called "lottery tickets" thatconverge quickly to a solution. To investigate how weight initializationsaffect performance, we examine small convolutional networks that are trained topredict n steps of the two-dimensional cellular automaton Conway's Game ofLife, the update rules of which can be implemented efficiently in a 2n+1 layerconvolutional network. We find that networks of this architecture trained onthis task rarely converge. Rather, networks require substantially moreparameters to consistently converge. In addition, near-minimal architecturesare sensitive to tiny changes in parameters: changing the sign of a singleweight can cause the network to fail to learn. Finally, we observe a criticalvalue d_0 such that training minimal networks with examples in which cells arealive with probability d_0 dramatically increases the chance of convergence toa solution. We conclude that training convolutional neural networks to learnthe input/output function represented by n steps of Game of Life exhibits manycharacteristics predicted by the lottery ticket hypothesis, namely, that thesize of the networks required to learn this function are often significantlylarger than the minimal network required to implement the function.

It's Hard for Neural Networks To Learn the Game of Life