English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While advanced classifiers have been increasingly used in real-worldsafety-critical applications, how to properly evaluate the black-box modelsgiven specific human values remains a concern in the community. Such humanvalues include punishing error cases of different severity in varying degreesand making compromises in general performance to reduce specific dangerouscases. In this paper, we propose a novel evaluation measure named Meta PatternConcern Score based on the abstract representation of probabilistic predictionand the adjustable threshold for the concession in prediction confidence, tointroduce the human values into multi-classifiers. Technically, we learn fromthe advantages and disadvantages of two kinds of common metrics, namely theconfusion matrix-based evaluation measures and the loss values, so that ourmeasure is effective as them even under general tasks, and the cross entropyloss becomes a special case of our measure in the limit. Besides, our measurecan also be used to refine the model training by dynamically adjusting thelearning rate. The experiments on four kinds of models and six datasets confirmthe effectiveness and efficiency of our measure. And a case study shows it cannot only find the ideal model reducing 0.53% of dangerous cases by onlysacrificing 0.04% of training accuracy, but also refine the learning rate totrain a new model averagely outperforming the original one with a 1.62% lowervalue of itself and 0.36% fewer number of dangerous cases.

Meta Pattern Concern Score: A Novel Evaluation Measure with Human Values for Multi-classifiers