English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The lottery ticket hypothesis (LTH) has attracted attention because it canexplain why over-parameterized models often show high generalization ability.It is known that when we use iterative magnitude pruning (IMP), which is analgorithm to find sparse networks with high generalization ability that can betrained from the initial weights independently, called winning tickets, theinitial large learning rate does not work well in deep neural networks such asResNet. However, since the initial large learning rate generally helps theoptimizer to converge to flatter minima, we hypothesize that the winningtickets have relatively sharp minima, which is considered a disadvantage interms of generalization ability. In this paper, we confirm this hypothesis andshow that the PAC-Bayesian theory can provide an explicit understanding of therelationship between LTH and generalization behavior. On the basis of ourexperimental findings that flatness is useful for improving accuracy androbustness to label noise and that the distance from the initial weights isdeeply involved in winning tickets, we offer the PAC-Bayes bound using aspike-and-slab distribution to analyze winning tickets. Finally, we revisitexisting algorithms for finding winning tickets from a PAC-Bayesian perspectiveand provide new insights into these methods.

Analyzing Lottery Ticket Hypothesis from PAC-Bayesian Theory Perspective