English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In a well-calibrated risk prediction model, the average predicted probabilityis close to the true event rate for any given subgroup. Such models arereliable across heterogeneous populations and satisfy strong notions ofalgorithmic fairness. However, the task of auditing a model for strongcalibration is well-known to be difficult -- particularly for machine learning(ML) algorithms -- due to the sheer number of potential subgroups. As such,common practice is to only assess calibration with respect to a few predefinedsubgroups. Recent developments in goodness-of-fit testing offer potentialsolutions but are not designed for settings with weak signal or where thepoorly calibrated subgroup is small, as they either overly subdivide the dataor fail to divide the data at all. We introduce a new testing procedure basedon the following insight: if we can reorder observations by their expectedresiduals, there should be a change in the association between the predictedand observed residuals along this sequence if a poorly calibrated subgroupexists. This lets us reframe the problem of calibration testing into one ofchangepoint detection, for which powerful methods already exist. We begin withintroducing a sample-splitting procedure where a portion of the data is used totrain a suite of candidate models for predicting the residual, and theremaining data are used to perform a score-based cumulative sum (CUSUM) test.To further improve power, we then extend this adaptive CUSUM test toincorporate cross-validation, while maintaining Type I error control underminimal assumptions. Compared to existing methods, the proposed procedureconsistently achieved higher power in simulation studies and more than doubledthe power when auditing a mortality risk prediction model.

Is this model reliable for everyone? Testing for strong calibration