English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We show that Entropy-SGD (Chaudhari et al., 2017), when viewed as a learningalgorithm, optimizes a PAC-Bayes bound on the risk of a Gibbs (posterior)classifier, i.e., a randomized classifier obtained by a risk-sensitiveperturbation of the weights of a learned classifier. Entropy-SGD works byoptimizing the bound's prior, violating the hypothesis of the PAC-Bayes theoremthat the prior is chosen independently of the data. Indeed, availableimplementations of Entropy-SGD rapidly obtain zero training error on randomlabels and the same holds of the Gibbs posterior. In order to obtain a validgeneralization bound, we rely on a result showing that data-dependent priorsobtained by stochastic gradient Langevin dynamics (SGLD) yield valid PAC-Bayesbounds provided the target distribution of SGLD is {\epsilon}-differentiallyprivate. We observe that test error on MNIST and CIFAR10 falls within the(empirically nonvacuous) risk bounds computed under the assumption that SGLDreaches stationarity. In particular, Entropy-SGLD can be configured to yieldrelatively tight generalization bounds and still fit real labels, althoughthese same settings do not obtain state-of-the-art performance.

Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priors