English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

There are strong incentives to build models that demonstrate outstandingpredictive performance on various datasets and benchmarks. We believe theseincentives risk a narrow focus on models and on the performance metrics used toevaluate and compare them -- resulting in a growing body of literature toevaluate and compare metrics. This paper strives for a more balancedperspective on classifier performance metrics by highlighting theirdistributions under different models of uncertainty and showing how thisuncertainty can easily eclipse differences in the empirical performance ofclassifiers. We begin by emphasising the fundamentally discrete nature ofempirical confusion matrices and show how binary matrices can be meaningfullyrepresented in a three dimensional compositional lattice, whose cross-sectionsform the basis of the space of receiver operating characteristic (ROC) curves.We develop equations, animations and interactive visualisations of the contoursof performance metrics within (and beyond) this ROC space, showing how some areaffected by class imbalance. We provide interactive visualisations that showthe discrete posterior predictive probability mass functions of true and falsepositive rates in ROC space, and how these relate to uncertainty in performancemetrics such as Balanced Accuracy (BA) and the Matthews Correlation Coefficient(MCC). Our hope is that these insights and visualisations will raise greaterawareness of the substantial uncertainty in performance metric estimates thatcan arise when classifiers are evaluated on empirical datasets and benchmarks,and that classification model performance claims should be tempered by thisunderstanding.

Never mind the metrics -- what about the uncertainty? Visualising confusion matrix metric distributions