English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Societal biases are reflected in large pre-trained language models and theirfine-tuned versions on downstream tasks. Common in-processing bias mitigationapproaches, such as adversarial training and mutual information removal,introduce additional optimization criteria, and update the model to reach a newdebiased state. However, in practice, end-users and practitioners might preferto switch back to the original model, or apply debiasing only on a specificsubset of protected attributes. To enable this, we propose a novel modular biasmitigation approach, consisting of stand-alone highly sparse debiasingsubnetworks, where each debiasing module can be integrated into the core modelon-demand at inference time. Our approach draws from the concept of \emph{diff}pruning, and proposes a novel training regime adaptable to variousrepresentation disentanglement optimizations. We conduct experiments on threeclassification tasks with gender, race, and age as protected attributes. Theresults show that our modular approach, while maintaining task performance,improves (or at least remains on-par with) the effectiveness of bias mitigationin comparison with baseline finetuning. Particularly on a two-attributedataset, our approach with separately learned debiasing subnetworks showseffective utilization of either or both the subnetworks for selective biasmitigation.

Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks