English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Multilingual pre-training significantly improves many multilingual NLP tasks,including machine translation. Most existing methods are based on some variantsof masked language modeling and text-denoising objectives on monolingual data.Multilingual pre-training on monolingual data ignores the availability ofparallel data in many language pairs. Also, some other works integrate theavailable human-generated parallel translation data in their pre-training. Thiskind of parallel data is definitely helpful, but it is limited even inhigh-resource language pairs. This paper introduces a novel semi-supervisedmethod, SPDG, that generates high-quality pseudo-parallel data for multilingualpre-training. First, a denoising model is pre-trained on monolingual data toreorder, add, remove, and substitute words, enhancing the pre-trainingdocuments' quality. Then, we generate different pseudo-translations for eachpre-training document using dictionaries for word-by-word translation andapplying the pre-trained denoising model. The resulting pseudo-parallel data isthen used to pre-train our multilingual sequence-to-sequence model, PEACH. Ourexperiments show that PEACH outperforms existing approaches used in trainingmT5 and mBART on various translation tasks, including supervised, zero- andfew-shot scenarios. Moreover, PEACH's ability to transfer knowledge betweensimilar languages makes it particularly useful for low-resource languages. Ourresults demonstrate that with high-quality dictionaries for generating accuratepseudo-parallel, PEACH can be valuable for low-resource languages.

PEACH: Pre-Training Sequence-to-Sequence Multilingual Models for Translation with Semi-Supervised Pseudo-Parallel Document Generation