English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

We present Nesterov-type acceleration techniques for Alternating LeastSquares (ALS) methods applied to canonical tensor decomposition. While Nesterovacceleration turns gradient descent into an optimal first-order method forconvex problems by adding a momentum term with a specific weight sequence, adirect application of this method and weight sequence to ALS results in erraticconvergence behaviour. This is so because the tensor decomposition problem isnon-convex and ALS is accelerated instead of gradient descent. Instead, weconsider various restart mechanisms and suitable choices of momentum weightsthat enable effective acceleration. Our extensive empirical results show thatthe Nesterov-accelerated ALS methods with restart can be dramatically moreefficient than the stand-alone ALS or Nesterov accelerated gradient methods,when problems are ill-conditioned or accurate solutions are desired. Theresulting methods perform competitively with or superior to existingacceleration methods for ALS, including ALS acceleration by NCG, NGMRES, orLBFGS, and additionally enjoy the benefit of being much easier to implement. Wealso compare with Nesterov-type updates where the momentum weight is determinedby a line search, which are equivalent or closely related to existing linesearch methods for ALS. On a large and ill-conditioned71$\times$1000$\times$900 tensor consisting of readings from chemical sensorsto track hazardous gases, the restarted Nesterov-ALS method shows desirablerobustness properties and outperforms any of the existing methods by a largefactor. There is clear potential for extending our Nesterov-type accelerationapproach to accelerating other optimization algorithms than ALS applied toother non-convex problems, such as Tucker tensor decomposition. Our Matlab codeis available athttps://github.com/hansdesterck/nonlinear-preconditioning-for-optimization.

Nesterov Acceleration of Alternating Least Squares for Canonical Tensor Decomposition: Momentum Step Size Selection and Restart Mechanisms