English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

As demand for large corpora increases with the size of currentstate-of-the-art language models, using web data as the main part of thepre-training corpus for these models has become a ubiquitous practice. This, inturn, has introduced an important challenge for NLP practitioners, as they arenow confronted with the task of developing highly optimized models andpipelines for pre-processing large quantities of textual data, which implies,effectively classifying and filtering multilingual, heterogeneous and noisydata, at web scale. One of the main components of this pre-processing step forthe pre-training corpora of large language models, is the removal of adult andharmful content. In this paper we explore different methods for detecting adultand harmful of content in multilingual heterogeneous web data. We first showhow traditional methods in harmful content detection, that seemingly performquite well in small and specialized datasets quickly break down when confrontedwith heterogeneous noisy web data. We then resort to using a perplexity basedapproach but with a twist: Instead of using a so-called "clean" corpus to traina small language model and then use perplexity so select the documents with lowperplexity, i.e., the documents that resemble this so-called "clean" corpus themost. We train solely with adult and harmful textual data, and then select thedocuments having a perplexity value above a given threshold. This approach willvirtually cluster our documents into two distinct groups, which will greatlyfacilitate the choice of the threshold for the perplexity and will also allowus to obtain higher precision than with the traditional classification methodsfor detecting adult and harmful content.

Perplexed by Quality: A Perplexity-based Method for Adult and Harmful Content Detection in Multilingual Heterogeneous Web Data