English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose a novel task-agnostic in-domain pre-training method that sitsbetween generic pre-training and fine-tuning. Our approach selectively masksin-domain keywords, i.e., words that provide a compact representation of thetarget domain. We identify such keywords using KeyBERT (Grootendorst, 2020). Weevaluate our approach using six different settings: three datasets combinedwith two distinct pre-trained language models (PLMs). Our results reveal thatthe fine-tuned PLMs adapted using our in-domain pre-training strategyoutperform PLMs that used in-domain pre-training with random masking as well asthose that followed the common pre-train-then-fine-tune paradigm. Further, theoverhead of identifying in-domain keywords is reasonable, e.g., 7-15% of thepre-training time (for two epochs) for BERT Large (Devlin et al., 2019).

Do not Mask Randomly: Effective Domain-adaptive Pre-training by Masking In-domain Keywords