English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The click-through rate (CTR) prediction task is to predict whether a userwill click on the recommended item. As mind-boggling amounts of data areproduced online daily, accelerating CTR prediction model training is criticalto ensuring an up-to-date model and reducing the training cost. One approach toincrease the training speed is to apply large batch training. However, as shownin computer vision and natural language processing tasks, training with a largebatch easily suffers from the loss of accuracy. Our experiments show thatprevious scaling rules fail in the training of CTR prediction neural networks.To tackle this problem, we first theoretically show that different frequenciesof ids make it challenging to scale hyperparameters when scaling the batchsize. To stabilize the training process in a large batch size setting, wedevelop the adaptive Column-wise Clipping (CowClip). It enables an easy andeffective scaling rule for the embeddings, which keeps the learning rateunchanged and scales the L2 loss. We conduct extensive experiments with fourCTR prediction networks on two real-world datasets and successfully scaled 128times the original batch size without accuracy loss. In particular, for CTRprediction model DeepFM training on the Criteo dataset, our optimizationframework enlarges the batch size from 1K to 128K with over 0.1% AUCimprovement and reduces training time from 12 hours to 10 minutes on a singleV100 GPU. Our code locates at https://github.com/bytedance/LargeBatchCTR.

CowClip: Reducing CTR Prediction Model Training Time from 12 hours to 10 minutes on 1 GPU