English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

This paper presents a new optimisation approach to train Deep Neural Networks(DNNs) with discriminative sequence criteria. At each iteration, the methodcombines information from the Natural Gradient (NG) direction with localcurvature information of the error surface that enables better paths on theparameter manifold to be traversed. The method is derived using an alternativederivation of Taylor's theorem using the concepts of manifolds, tangent vectorsand directional derivatives from the perspective of Information Geometry. Theefficacy of the method is shown within a Hessian Free (HF) style optimisationframework to sequence train both standard fully-connected DNNs and Time DelayNeural Networks as speech recognition acoustic models. It is shown that for thesame number of updates the proposed approach achieves larger reductions in theword error rate (WER) than both NG and HF, and also leads to a lower WER thanstandard stochastic gradient descent. The paper also addresses the issue ofover-fitting due to mismatch between training criterion and Word Error Rate(WER) that primarily arises during sequence training of ReLU-DNN models.

Combining Natural Gradient with Hessian Free Methods for Sequence Training