Learning Dictionaries of Discriminative Image Patches

Remarkable results have been obtained using image models based on image patches, for example sparse generative models for image inpainting, noise reduction and superresolution, sparse texture segmentation or texton models. In this paper we propose a powerful and yet simple approach for segmentation using dictionaries of image patches with associated label data. The approach is based on ideas from sparse generative image models and texton based texture modeling. The intensity and label dictionaries are learned from training images with associated label information of (a subset) of the pixels based on a modified vector quantization approach. For new images the intensity dictionary is used to encode the image data and the label dictionary is used to build a segmentation of the image. We demonstrate the algorithm on composite and real texture images and show how successful training is possible even for noisy image and low-quality label training data. In our experimental evaluation we achieve state-of-the-art performance for segmentation.