English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep learning recommendation systems at scale have provided remarkable gainsthrough increasing model capacity (i.e. wider and deeper neural networks), butit comes at significant training cost and infrastructure cost. Model pruning isan effective technique to reduce computation overhead for deep neural networksby removing redundant parameters. However, modern recommendation systems arestill thirsty for model capacity due to the demand for handling big data. Thus,pruning a recommendation model at scale results in a smaller model capacity andconsequently lower accuracy. To reduce computation cost without sacrificingmodel capacity, we propose a dynamic training scheme, namely alternate modelgrowth and pruning, to alternatively construct and prune weights in the courseof training. Our method leverages structured sparsification to reducecomputational cost without hurting the model capacity at the end of offlinetraining so that a full-size model is available in the recurring training stageto learn new data in real-time. To the best of our knowledge, this is the firstwork to provide in-depth experiments and discussion of applying structuraldynamics to recommendation systems at scale to reduce training cost. Theproposed method is validated with an open-source deep-learning recommendationmodel (DLRM) and state-of-the-art industrial-scale production models.

Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems