English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Loss landscape analysis is extremely useful for a deeper understanding of thegeneralization ability of deep neural network models. In this work, we proposea layerwise loss landscape analysis where the loss surface at every layer isstudied independently and also on how each correlates to the overall losssurface. We study the layerwise loss landscape by studying the eigenspectra ofthe Hessian at each layer. In particular, our results show that the layerwiseHessian geometry is largely similar to the entire Hessian. We also report aninteresting phenomenon where the Hessian eigenspectrum of middle layers of thedeep neural network are observed to most similar to the overall Hessianeigenspectrum. We also show that the maximum eigenvalue and the trace of theHessian (both full network and layerwise) reduce as training of the networkprogresses. We leverage on these observations to propose a new regularizerbased on the trace of the layerwise Hessian. Penalizing the trace of theHessian at every layer indirectly forces Stochastic Gradient Descent toconverge to flatter minima, which are shown to have better generalizationperformance. In particular, we show that such a layerwise regularizer can beleveraged to penalize the middlemost layers alone, which yields promisingresults. Our empirical studies on well-known deep nets across datasets supportthe claims of this work

A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to Regularization