Marginal Debiased Network for Fair Visual Recognition

Deep neural networks (DNNs) are often prone to learn the spuriouscorrelations between target classes and bias attributes, like gender and race,inherent in a major portion of training data (bias-aligned samples), thusshowing unfair behavior and arising controversy in the modern pluralistic andegalitarian society. In this paper, we propose a novel marginal debiasednetwork (MDN) to learn debiased representations. More specifically, a marginalsoftmax loss (MSL) is designed by introducing the idea of margin penalty intothe fairness problem, which assigns a larger margin for bias-conflictingsamples (data without spurious correlations) than for bias-aligned ones, so asto deemphasize the spurious correlations and improve generalization on unbiasedtest criteria. To determine the margins, our MDN is optimized through a metalearning framework. We propose a meta equalized loss (MEL) to perceive themodel fairness, and adaptively update the margin parameters by metaoptimizationwhich requires the trained model guided by the optimal margins should minimizeMEL computed on an unbiased meta-validation set. Extensive experiments onBiasedMNIST, Corrupted CIFAR-10, CelebA and UTK-Face datasets demonstrate thatour MDN can achieve a remarkable performance on under-represented samples andobtain superior debiased results against the previous approaches.