English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Artificial writing is permeating our lives due to recent advances inlarge-scale, transformer-based language models (LMs) such as BERT, itsvariants, GPT-2/3, and others. Using them as pre-trained models and fine-tuningthem for specific tasks, researchers have extended state of the art for manyNLP tasks and shown that they capture not only linguistic knowledge but alsoretain general knowledge implicitly present in the data. Unfortunately, LMstrained on unfiltered text corpora suffer from degenerated and biasedbehaviour. While this is well established, we show that recent LMs also containhuman-like biases of what is right and wrong to do, some form of ethical andmoral norms of the society -- they bring a "moral direction" to surface. Thatis, we show that these norms can be captured geometrically by a direction,which can be computed, e.g., by a PCA, in the embedding space, reflecting wellthe agreement of phrases to social norms implicitly expressed in the trainingtexts and providing a path for attenuating or even preventing toxicdegeneration in LMs. Being able to rate the (non-)normativity of arbitraryphrases without explicitly training the LM for this task, we demonstrate thecapabilities of the "moral direction" for guiding (even other) LMs towardsproducing normative text and showcase it on RealToxicityPrompts testbed,preventing the neural toxic degeneration in GPT-2.

Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do