English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-trained language models such as BERT have exhibited remarkableperformances in many tasks in natural language understanding (NLU). The tokensin the models are usually fine-grained in the sense that for languages likeEnglish they are words or sub-words and for languages like Chinese they arecharacters. In English, for example, there are multi-word expressions whichform natural lexical units and thus the use of coarse-grained tokenization alsoappears to be reasonable. In fact, both fine-grained and coarse-grainedtokenizations have advantages and disadvantages for learning of pre-trainedlanguage models. In this paper, we propose a novel pre-trained language model,referred to as AMBERT (A Multi-grained BERT), on the basis of both fine-grainedand coarse-grained tokenizations. For English, AMBERT takes both the sequenceof words (fine-grained tokens) and the sequence of phrases (coarse-grainedtokens) as input after tokenization, employs one encoder for processing thesequence of words and the other encoder for processing the sequence of thephrases, utilizes shared parameters between the two encoders, and finallycreates a sequence of contextualized representations of the words and asequence of contextualized representations of the phrases. Experiments havebeen conducted on benchmark datasets for Chinese and English, including CLUE,GLUE, SQuAD and RACE. The results show that AMBERT can outperform BERT in allcases, particularly the improvements are significant for Chinese. We alsodevelop a method to improve the efficiency of AMBERT in inference, which stillperforms better than BERT with the same computational cost as BERT.

AMBERT: A Pre-trained Language Model with Multi-Grained Tokenization