English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Monitoring machine learning models once they are deployed is challenging. Itis even more challenging to decide when to retrain models in real-casescenarios when labeled data is beyond reach, and monitoring performance metricsbecomes unfeasible. In this work, we use non-parametric bootstrappeduncertainty estimates and SHAP values to provide explainable uncertaintyestimation as a technique that aims to monitor the deterioration of machinelearning models in deployment environments, as well as determine the source ofmodel deterioration when target labels are not available. Classical methods arepurely aimed at detecting distribution shift, which can lead to false positivesin the sense that the model has not deteriorated despite a shift in the datadistribution. To estimate model uncertainty we construct prediction intervalsusing a novel bootstrap method, which improves upon the work of Kumar &Srivastava (2012). We show that both our model deterioration detection systemas well as our uncertainty estimation method achieve better performance thanthe current state-of-the-art. Finally, we use explainable AI techniques to gainan understanding of the drivers of model deterioration. We release an opensource Python package, doubt, which implements our proposed methods, as well asthe code used to reproduce our experiments.

Monitoring Model Deterioration with Explainable Uncertainty Estimation via Non-parametric Bootstrap