English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With the ever-growing presence of social media platforms comes the increasedspread of harmful content and the need for robust hate speech detectionsystems. Such systems easily overfit to specific targets and keywords, andevaluating them without considering distribution shifts that might occurbetween train and test data overestimates their benefit. We challenge hatespeech models via new train-test splits of existing datasets that rely on theclustering of models' hidden representations. We present two split variants(Subset-Sum-Split and Closest-Split) that, when applied to two datasets usingfour pretrained models, reveal how models catastrophically fail on blind spotsin the latent space. This result generalises when developing a split with onemodel and evaluating it on another. Our analysis suggests that there is noclear surface-level property of the data split that correlates with thedecreased performance, which underscores that task difficulty is not alwayshumanly interpretable. We recommend incorporating latent feature-based splitsin model development and release two splits via the GenBench benchmark.

Latent Feature-based Data Splits to Improve Generalisation Evaluation: A Hate Speech Detection Case Study