English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-trained language models (PLMs) like BERT have made significant progressin various downstream NLP tasks. However, by asking models to do cloze-styletests, recent work finds that PLMs are short in acquiring knowledge fromunstructured text. To understand the internal behaviour of PLMs in retrievingknowledge, we first define knowledge-baring (K-B) tokens and knowledge-free(K-F) tokens for unstructured text and ask professional annotators to labelsome samples manually. Then, we find that PLMs are more likely to give wrongpredictions on K-B tokens and attend less attention to those tokens inside theself-attention module. Based on these observations, we develop two solutions tohelp the model learn more knowledge from unstructured text in a fullyself-supervised manner. Experiments on knowledge-intensive tasks show theeffectiveness of the proposed methods. To our best knowledge, we are the firstto explore fully self-supervised learning of knowledge in continualpre-training.

Knowledgeable Salient Span Mask for Enhancing Language Models as Knowledge Base