English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

A critical problem in deep learning is that systems learn inappropriatebiases, resulting in their inability to perform well on minority groups. Thishas led to the creation of multiple algorithms that endeavor to mitigate bias.However, it is not clear how effective these methods are. This is because studyprotocols differ among papers, systems are tested on datasets that fail to testmany forms of bias, and systems have access to hidden knowledge or are tunedspecifically to the test set. To address this, we introduce an improvedevaluation protocol, sensible metrics, and a new dataset, which enables us toask and answer critical questions about bias mitigation algorithms. We evaluateseven state-of-the-art algorithms using the same network architecture andhyperparameter selection policy across three benchmark datasets. We introduce anew dataset called Biased MNIST that enables assessment of robustness tomultiple bias sources. We use Biased MNIST and a visual question answering(VQA) benchmark to assess robustness to hidden biases. Rather than only tuningto the test set distribution, we study robustness across different tuningdistributions, which is critical because for many applications the testdistribution may not be known during development. We find that algorithmsexploit hidden biases, are unable to scale to multiple forms of bias, and arehighly sensitive to the choice of tuning set. Based on our findings, we implorethe community to adopt more rigorous assessment of future bias mitigationmethods. All data, code, and results are publicly available at:https://github.com/erobic/bias-mitigators.

Are Bias Mitigation Techniques for Deep Learning Effective?