English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Underspecification and fairness in machine learning (ML) applications haverecently become two prominent issues in the ML community. Acoustic sceneclassification (ASC) applications have so far remained unaffected by thisdiscussion, but are now becoming increasingly used in real-world systems wherefairness and reliability are critical aspects. In this work, we argue for theneed of a more holistic evaluation process for ASC models through disaggregatedevaluations. This entails taking into account performance differences acrossseveral factors, such as city, location, and recording device. Although thesefactors play a well-understood role in the performance of ASC models, mostworks report single evaluation metrics taking into account all different strataof a particular dataset. We argue that metrics computed on specificsub-populations of the underlying data contain valuable information about theexpected real-world behaviour of proposed systems, and their reporting couldimprove the transparency and trustability of such systems. We demonstrate theeffectiveness of the proposed evaluation process in uncoveringunderspecification and fairness problems exhibited by several standard MLarchitectures when trained on two widely-used ASC datasets. Our evaluationshows that all examined architectures exhibit large biases across all factorstaken into consideration, and in particular with respect to the recordinglocation. Additionally, different architectures exhibit different biases eventhough they are trained with the same experimental configurations.

Fairness and underspecification in acoustic scene classification: The case for disaggregated evaluations