English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Machine learning models are increasingly used in practice. However, manymachine learning methods are sensitive to test or operational data that isdissimilar to training data. Out-of-distribution (OOD) data is known toincrease the probability of error and research into metrics that identify whatdissimilarities in data affect model performance is on-going. Recently,combinatorial coverage metrics have been explored in the literature as analternative to distribution-based metrics. Results show that coverage metricscan correlate with classification error. However, other results show that theutility of coverage metrics is highly dataset-dependent. In this paper, we showthat this dataset-dependence can be alleviated with metric learning, a machinelearning technique for learning latent spaces where data from different classesis further apart. In a study of 6 open-source datasets, we find that metriclearning increased the difference between set-difference coverage metrics(SDCCMs) calculated on correctly and incorrectly classified data, therebydemonstrating that metric learning improves the ability of SDCCMs to anticipateclassification error. Paired t-tests validate the statistical significance ofour findings. Overall, we conclude that metric learning improves the ability ofcoverage metrics to anticipate classifier error and identify when OOD data islikely to degrade model performance.

Metric Learning Improves the Ability of Combinatorial Coverage Metrics to Anticipate Classification Error