English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-trained language models (PLMs) like BERT have made great progress in NLP.News articles usually contain rich textual information, and PLMs have thepotentials to enhance news text modeling for various intelligent newsapplications like news recommendation and retrieval. However, most existingPLMs are in huge size with hundreds of millions of parameters. Many online newsapplications need to serve millions of users with low latency tolerance, whichposes huge challenges to incorporating PLMs in these scenarios. Knowledgedistillation techniques can compress a large PLM into a much smaller one andmeanwhile keeps good performance. However, existing language models arepre-trained and distilled on general corpus like Wikipedia, which has some gapswith the news domain and may be suboptimal for news intelligence. In thispaper, we propose NewsBERT, which can distill PLMs for efficient and effectivenews intelligence. In our approach, we design a teacher-student joint learningand distillation framework to collaboratively learn both teacher and studentmodels, where the student model can learn from the learning experience of theteacher model. In addition, we propose a momentum distillation method byincorporating the gradients of teacher model into the update of student modelto better transfer useful knowledge learned by the teacher model. Extensiveexperiments on two real-world datasets with three tasks show that NewsBERT caneffectively improve the model performance in various intelligent newsapplications with much smaller models.

NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application