Generative Models for Labeling Multi-object Configurations in Images

We propose a generative approach to the problem of label- ing images containing congurations of objects from multiple classes. The main building blocks are dense statistical models for individual ob- jects. The models assume conditional independence of binary oriented edge variables conditional on a hidden instantiation parameter, which also determines an object support. These models are then be composed to form models for object congurations with various interactions includ- ing occlusion. Choosing the optimal conguration is entirely likelihood based and no decision boundaries need to be pre-learned. Training in- volves estimation of model parameters for each class separately. Both training and classication involve estimation of hidden pose variables which can be computationally intensive. We describe two levels of ap- proximation which facilitate these computations: the Patchwork of Parts (POP) model and the coarse part based models (CPM). A concrete im- plementation of the approach is illustrated on the problem of reading zip-codes.