English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Lottery tickets (LTs) is able to discover accurate and sparse subnetworksthat could be trained in isolation to match the performance of dense networks.Ensemble, in parallel, is one of the oldest time-proven tricks in machinelearning to improve performance by combining the output of multiple independentmodels. However, the benefits of ensemble in the context of LTs will be dilutedsince ensemble does not directly lead to stronger sparse subnetworks, butleverages their predictions for a better decision. In this work, we firstobserve that directly averaging the weights of the adjacent learned subnetworkssignificantly boosts the performance of LTs. Encouraged by this observation, wefurther propose an alternative way to perform an 'ensemble' over thesubnetworks identified by iterative magnitude pruning via a simpleinterpolating strategy. We call our method Lottery Pools. In contrast to thenaive ensemble which brings no performance gains to each single subnetwork,Lottery Pools yields much stronger sparse subnetworks than the original LTswithout requiring any extra training or inference cost. Across various modernarchitectures on CIFAR-10/100 and ImageNet, we show that our method achievessignificant performance gains in both, in-distribution and out-of-distributionscenarios. Impressively, evaluated with VGG-16 and ResNet-18, the producedsparse subnetworks outperform the original LTs by up to 1.88% on CIFAR-100 and2.36% on CIFAR-100-C; the resulting dense network surpasses the pre-traineddense-model up to 2.22% on CIFAR-100 and 2.38% on CIFAR-100-C.

Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost