English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Language models excel at generating coherent text, and model compressiontechniques such as knowledge distillation have enabled their use inresource-constrained settings. However, these models can be biased in multipleways, including the unfounded association of male and female genders withgender-neutral professions. Therefore, knowledge distillation without anyfairness constraints may preserve or exaggerate the teacher model's biases ontothe distilled model. To this end, we present a novel approach to mitigategender disparity in text generation by learning a fair model during knowledgedistillation. We propose two modifications to the base knowledge distillationbased on counterfactual role reversal$\unicode{x2014}$modifying teacherprobabilities and augmenting the training set. We evaluate gender polarityacross professions in open-ended text generated from the resulting distilledand finetuned GPT$\unicode{x2012}$2 models and demonstrate a substantialreduction in gender disparity with only a minor compromise in utility. Finally,we observe that language models that reduce gender polarity in languagegeneration do not improve embedding fairness or downstream classificationfairness.

Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal