English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The success of DNNs is driven by the counter-intuitive ability ofover-parameterized networks to generalize, even when they perfectly fit thetraining data. In practice, test error often continues to decrease withincreasing over-parameterization, referred to as double descent. This allowspractitioners to instantiate large models without having to worry aboutover-fitting. Despite its benefits, however, prior work has shown thatover-parameterization can exacerbate bias against minority subgroups. Severalfairness-constrained DNN training methods have been proposed to address thisconcern. Here, we critically examine MinDiff, a fairness-constrained trainingprocedure implemented within TensorFlow's Responsible AI Toolkit, that aims toachieve Equality of Opportunity. We show that although MinDiff improvesfairness for under-parameterized models, it is likely to be ineffective in theover-parameterized regime. This is because an overfit model with zero trainingloss is trivially group-wise fair on training data, creating an "illusion offairness," thus turning off the MinDiff optimization (this will apply to anydisparity-based measures which care about errors or accuracy. It won't apply todemographic parity). Within specified fairness constraints, under-parameterizedMinDiff models can even have lower error compared to their over-parameterizedcounterparts (despite baseline over-parameterized models having lower error).We further show that MinDiff optimization is very sensitive to choice of batchsize in the under-parameterized regime. Thus, fair model training using MinDiffrequires time-consuming hyper-parameter searches. Finally, we suggest usingpreviously proposed regularization techniques, viz. L2, early stopping andflooding in conjunction with MinDiff to train fair over-parameterized models.

Fairness via In-Processing in the Over-parameterized Regime: A Cautionary Tale