English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We investigate a sequential optimization procedure to minimize the empiricalrisk functional $f_{\hat\theta}(x) = \frac{1}{2}\|G_{\hat\theta}(x) - y\|^2$for certain families of deep networks $G_{\theta}(x)$. The approach is tooptimize a sequence of objective functions that use network parameters obtainedduring different stages of the training process. When initialized with randomparameters $\theta_0$, we show that the objective $f_{\theta_0}(x)$ is "nice''and easy to optimize with gradient descent. As learning is carried out, weobtain a sequence of generative networks $x \mapsto G_{\theta_t}(x)$ andassociated risk functions $f_{\theta_t}(x)$, where $t$ indicates a stage ofstochastic gradient descent during training. Since the parameters of thenetwork do not change by very much in each step, the surface evolves slowly andcan be incrementally optimized. The algorithm is formalized and analyzed for afamily of expansive networks. We call the procedure {\it surfing} since itrides along the peak of the evolving (negative) empirical risk function,starting from a smooth surface at the beginning of learning and ending with awavy nonconvex surface after learning is complete. Experiments show how surfingcan be used to find the global optimum and for compressed sensing even whendirect gradient descent on the final learned network fails.

Surfing: Iterative optimization over incrementally trained deep networks