English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose to use stochastic Riemannian coordinate descent on the orthogonalgroup for recurrent neural network training. The algorithm rotates successivelytwo columns of the recurrent matrix, an operation that can be efficientlyimplemented as a multiplication by a Givens matrix. In the case when thecoordinate is selected uniformly at random at each iteration, we prove theconvergence of the proposed algorithm under standard assumptions on the lossfunction, stepsize and minibatch noise. In addition, we numerically demonstratethat the Riemannian gradient in recurrent neural network training has anapproximately sparse structure. Leveraging this observation, we propose afaster variant of the proposed algorithm that relies on the Gauss-Southwellrule. Experiments on a benchmark recurrent neural network training problem arepresented to demonstrate the effectiveness of the proposed algorithm.

Coordinate descent on the orthogonal group for recurrent neural network training