English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

While large scale pre-trained language models such as BERT have achievedgreat success on various natural language understanding tasks, how toefficiently and effectively incorporate them into sequence-to-sequence modelsand the corresponding text generation tasks remains a non-trivial problem. Inthis paper, we propose to address this problem by taking two different BERTmodels as the encoder and decoder respectively, and fine-tuning them byintroducing simple and lightweight adapter modules, which are inserted betweenBERT layers and tuned on the task-specific dataset. In this way, we obtain aflexible and efficient model which is able to jointly leverage the informationcontained in the source-side and target-side BERT models, while bypassing thecatastrophic forgetting problem. Each component in the framework can beconsidered as a plug-in unit, making the framework flexible and task agnostic.Our framework is based on a parallel sequence decoding algorithm namedMask-Predict considering the bi-directional and conditional independent natureof BERT, and can be adapted to traditional autoregressive decoding easily. Weconduct extensive experiments on neural machine translation tasks where theproposed method consistently outperforms autoregressive baselines whilereducing the inference latency by half, and achieves $36.49$/$33.57$ BLEUscores on IWSLT14 German-English/WMT14 German-English translation. When adaptedto autoregressive decoding, the proposed method achieves $30.60$/$43.56$ BLEUscores on WMT14 English-German/English-French translation, on par with thestate-of-the-art baseline models.

Incorporating BERT into Parallel Sequence Decoding with Adapters