Generating fuzzy semantic metadata describing spatial relations from images using the R-histogram

Automatic generation of semantic metadata describing spatial relations is highly desirable for image digital libraries. Relative spatial relations between objects in an image convey important information about the image. Because the perception of spatial relations is subjective, we propose a novel framework for automatic metadata generation based on fuzzy k-NN classification that generates fuzzy semantic metadata describing spatial relations between objects in an image. For each pair of objects of interest, the corresponding R-Histogram is computed and used as input for a set of fuzzy k-NN classifiers. The R-Histogram is a quantitative representation of spatial relations between two objects. The outputs of the classifiers are soft class labels for each of the following eight spatial relations: 1) LEFT OF, 2) RIGHT OF, 3) ABOVE, 4) BELOW, 5) NEAR, 6) FAR, 7) INSIDE, 8) OUTSIDE. Because the classifier-training stage involves annotating the training images manually, it is desirable to use as few training images as possible. To address this issue, we applied existing prototype selection techniques and also devised two new extensions. We evaluated the performance of different fuzzy k-NN algorithms and prototype selection algorithms empirically on both synthetic and real images. Preliminary experimental results show that our system is able to obtain good annotation accuracy (92%-98% on synthetic images and 82%-93% on real images) using only a small training set (4-5 images).