U-Trustworthy Models.Reliability, Competence, and Confidence in Decision-Making

With growing concerns regarding bias and discrimination in predictive models,the AI community has increasingly focused on assessing AI systemtrustworthiness. Conventionally, trustworthy AI literature relies on theprobabilistic framework and calibration as prerequisites for trustworthiness.In this work, we depart from this viewpoint by proposing a novel trustframework inspired by the philosophy literature on trust. We present a precisemathematical definition of trustworthiness, termed$\mathcal{U}$-trustworthiness, specifically tailored for a subset of tasksaimed at maximizing a utility function. We argue that a model's$\mathcal{U}$-trustworthiness is contingent upon its ability to maximize Bayesutility within this task subset. Our first set of results challenges theprobabilistic framework by demonstrating its potential to favor lesstrustworthy models and introduce the risk of misleading trustworthinessassessments. Within the context of $\mathcal{U}$-trustworthiness, we prove thatproperly-ranked models are inherently $\mathcal{U}$-trustworthy. Furthermore,we advocate for the adoption of the AUC metric as the preferred measure oftrustworthiness. By offering both theoretical guarantees and experimentalvalidation, AUC enables robust evaluation of trustworthiness, thereby enhancingmodel selection and hyperparameter tuning to yield more trustworthy outcomes.