English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Machine learning models that achieve high overall accuracy often makesystematic errors on important subsets (or slices) of data. Identifyingunderperforming slices is particularly challenging when working withhigh-dimensional inputs (e.g. images, audio), where important slices are oftenunlabeled. In order to address this issue, recent studies have proposedautomated slice discovery methods (SDMs), which leverage learned modelrepresentations to mine input data for slices on which a model performs poorly.To be useful to a practitioner, these methods must identify slices that areboth underperforming and coherent (i.e. united by a human-understandableconcept). However, no quantitative evaluation framework currently exists forrigorously assessing SDMs with respect to these criteria. Additionally, priorqualitative evaluations have shown that SDMs often identify slices that areincoherent. In this work, we address these challenges by first designing aprincipled evaluation framework that enables a quantitative comparison of SDMsacross 1,235 slice discovery settings in three input domains (natural images,medical images, and time-series data). Then, motivated by the recentdevelopment of powerful cross-modal representation learning approaches, wepresent Domino, an SDM that leverages cross-modal embeddings and a novelerror-aware mixture model to discover and describe coherent slices. We findthat Domino accurately identifies 36% of the 1,235 slices in our framework - a12 percentage point improvement over prior methods. Further, Domino is thefirst SDM that can provide natural language descriptions of identified slices,correctly generating the exact name of the slice in 35% of settings.

Domino: Discovering Systematic Errors with Cross-Modal Embeddings