English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The abundance of annotated data in natural language processing (NLP) posesboth opportunities and challenges. While it enables the development ofhigh-performing models for a variety of tasks, it also poses the risk of modelslearning harmful biases from the data, such as gender stereotypes. In thiswork, we investigate the role of attention, a widely-used technique in currentstate-of-the-art NLP models, in the propagation of social biases. Specifically,we study the relationship between the entropy of the attention distribution andthe model's performance and fairness. We then propose a novel method formodulating attention weights to improve model fairness after training. Sinceour method is only applied post-training and pre-inference, it is anintra-processing method and is, therefore, less computationally expensive thanexisting in-processing and pre-processing approaches. Our results show anincrease in fairness and minimal performance loss on different textclassification and generation tasks using language models of varying sizes.WARNING: This work uses language that is offensive.

Should We Attend More or Less? Modulating Attention for Fairness