English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Pretrained language models have been used in various natural languageprocessing applications. In the mental health domain, domain-specific languagemodels are pretrained and released, which facilitates the early detection ofmental health conditions. Social posts, e.g., on Reddit, are usually longdocuments. However, there are no domain-specific pretrained models forlong-sequence modeling in the mental health domain. This paper conductsdomain-specific continued pretraining to capture the long context for mentalhealth. Specifically, we train and release MentalXLNet and MentalLongformerbased on XLNet and Longformer. We evaluate the mental health classificationperformance and the long-range ability of these two domain-specific pretrainedmodels. Our models are released in HuggingFace.

Domain-specific Continued Pretraining of Language Models for Capturing Long Context in Mental Health