English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

This work presents our ongoing research of unsupervised pretraining in neuralmachine translation (NMT). In our method, we initialize the weights of theencoder and decoder with two language models that are trained with monolingualdata and then fine-tune the model on parallel data using Elastic WeightConsolidation (EWC) to avoid forgetting of the original language modelingtasks. We compare the regularization by EWC with the previous work that focuseson regularization by language modeling objectives. The positive result is thatusing EWC with the decoder achieves BLEU scores similar to the previous work.However, the model converges 2-3 times faster and does not require the originalunlabeled training data during the fine-tuning stage. In contrast, theregularization using EWC is less effective if the original and new tasks arenot closely related. We show that initializing the bidirectional NMT encoderwith a left-to-right language model and forcing the model to remember theoriginal left-to-right language modeling task limits the learning capacity ofthe encoder for the whole bidirectional context.

Unsupervised Pretraining for Neural Machine Translation Using Elastic Weight Consolidation