English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

As machine learning (ML) systems increasingly permeate high-stakes settingssuch as healthcare, transportation, military, and national security, concernsregarding their reliability have emerged. Despite notable progress, theperformance of these systems can significantly diminish due to adversarialattacks or environmental changes, leading to overconfident predictions,failures to detect input faults, and an inability to generalize in unexpectedscenarios. This paper proposes a holistic assessment methodology for thereliability of ML systems. Our framework evaluates five key properties:in-distribution accuracy, distribution-shift robustness, adversarialrobustness, calibration, and out-of-distribution detection. A reliability scoreis also introduced and used to assess the overall system reliability. Toprovide insights into the performance of different algorithmic approaches, weidentify and categorize state-of-the-art techniques, then evaluate a selectionon real-world tasks using our proposed reliability metrics and reliabilityscore. Our analysis of over 500 models reveals that designing for one metricdoes not necessarily constrain others but certain algorithmic techniques canimprove reliability across multiple metrics simultaneously. This studycontributes to a more comprehensive understanding of ML reliability andprovides a roadmap for future research and development.

A Holistic Assessment of the Reliability of Machine Learning Systems