English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

ML models often exhibit unexpectedly poor behavior when they are deployed inreal-world domains. We identify underspecification as a key reason for thesefailures. An ML pipeline is underspecified when it can return many predictorswith equivalently strong held-out performance in the training domain.Underspecification is common in modern ML pipelines, such as those based ondeep learning. Predictors returned by underspecified pipelines are oftentreated as equivalent based on their training domain performance, but we showhere that such predictors can behave very differently in deployment domains.This ambiguity can lead to instability and poor model behavior in practice, andis a distinct failure mode from previously identified issues arising fromstructural mismatch between training and deployment domains. We show that thisproblem appears in a wide variety of practical ML pipelines, using examplesfrom computer vision, medical imaging, natural language processing, clinicalrisk prediction based on electronic health records, and medical genomics. Ourresults show the need to explicitly account for underspecification in modelingpipelines that are intended for real-world deployment in any domain.

Underspecification Presents Challenges for Credibility in Modern Machine Learning