English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

As the representation capability of Pre-trained Language Models (PLMs)improve, there is growing concern that they will inherit social biases fromunprocessed corpora. Most previous debiasing techniques used CounterfactualData Augmentation (CDA) to balance the training corpus. However, CDA slightlymodifies the original corpus, limiting the representation distance betweendifferent demographic groups to a narrow range. As a result, the debiasingmodel easily fits the differences between counterfactual pairs, which affectsits debiasing performance with limited text resources. In this paper, wepropose an adversarial training-inspired two-stage debiasing model usingContrastive learning with Continuous Prompt Augmentation (named CCPA) tomitigate social biases in PLMs' encoding. In the first stage, we propose a dataaugmentation method based on continuous prompt tuning to push farther therepresentation distance between sample pairs along different demographicgroups. In the second stage, we utilize contrastive learning to pull closer therepresentation distance between the augmented sample pairs and then fine-tunePLMs' parameters to get debiased encoding. Our approach guides the model toachieve stronger debiasing performance by adding difficulty to the trainingprocess. Extensive experiments show that CCPA outperforms baselines in terms ofdebiasing performance. Meanwhile, experimental results on the GLUE benchmarkshow that CCPA retains the language modeling capability of PLMs.

Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biases