Learning Directional Local Pairwise Bases with Sparse Coding

Recently, sparse coding has been receiving much attention in object and scene recognition tasks because of its superiority in learning an effective codebook over k-means clustering. However, empirically, such codebook requires a relatively large number of visual words, essentially bases, to achieve high recognition accuracy. Therefore, due to the combinatorial explosion of visual words, it is infeasible to use this codebook to represent higher-order spatial features which are equally important in capturing distinct properties of scenes and objects. Contrasted with many previous techniques that exploit higher-order spatial features, Local Pairwise Codebook (LPC) is a simple and effective method to learn a compact set of clusters representing pairs of spatially close descriptors with k-means. Based on LPC, this paper proposes Directional Local Pairwise Bases (DLPB) that applies sparse coding to learn a compact set of bases capturing correlation between these descriptors, so to avoid the combinatorial explosion. Furthermore, such bases are learned for each quantized direction thereby explicitly adding directional information to the representation. We have evaluated DLPB with several challenging object and scene category datasets. Our experimental results show that DLPB outperforms the baselines across all datasets and achieves the state-of-the-art performance on some datasets.