English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the statistical properties of learning to defer (L2D) to multipleexperts. In particular, we address the open problems of deriving a consistentsurrogate loss, confidence calibration, and principled ensembling of experts.Firstly, we derive two consistent surrogates -- one based on a softmaxparameterization, the other on a one-vs-all (OvA) parameterization -- that areanalogous to the single expert losses proposed by Mozannar and Sontag (2020)and Verma and Nalisnick (2022), respectively. We then study the frameworks'ability to estimate P( m_j = y | x ), the probability that the jth expert willcorrectly predict the label for x. Theory shows the softmax-based loss causesmis-calibration to propagate between the estimates while the OvA-based lossdoes not (though in practice, we find there are trade offs). Lastly, we proposea conformal inference technique that chooses a subset of experts to query whenthe system defers. We perform empirical validation on tasks for galaxy, skinlesion, and hate speech classification.

Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles