On surrogate loss functions and f-divergences

The goal of binary classification is to estimate a discriminant function$\gamma$ from observations of covariate vectors and corresponding binarylabels. We consider an elaboration of this problem in which the covariates arenot available directly but are transformed by a dimensionality-reducingquantizer $Q$. We present conditions on loss functions such that empirical riskminimization yields Bayes consistency when both the discriminant function andthe quantizer are estimated. These conditions are stated in terms of a generalcorrespondence between loss functions and a class of functionals known asAli-Silvey or $f$-divergence functionals. Whereas this correspondence wasestablished by Blackwell [Proc. 2nd Berkeley Symp. Probab. Statist. 1 (1951)93--102. Univ. California Press, Berkeley] for the 0--1 loss, we extend thecorrespondence to the broader class of surrogate loss functions that play a keyrole in the general theory of Bayes consistency for binary classification. Ourresult makes it possible to pick out the (strict) subset of surrogate lossfunctions that yield Bayes consistency for joint estimation of the discriminantfunction and the quantizer.