From Class-Specific to Class-Mixture: Cascaded Feature Representations via Restricted Boltzmann Machine Learning

In this paper, we propose two kinds of feature extracting frameworks that can extract cascaded class-specific and class-mixture features, respectively, by taking the restricted Boltzmann machine (RBM) as the basic building blocks; we further call them as a CS-RBM and CM-RBM feature extractor. The discriminations of features from both CS-RBM and CM-RBM are verified better than the class-independent (traditional) RBM (CI-RBM) feature extractor. As one mini-batch samples are randomly selected from all classes during the training phase of the traditional RBM, which can make that the above mini-batch data contain easy-confusing samples from different categories. Therefore, the features from CI-RBM are difficult to distinguish these samples from the confused categories. CS-RBM and CM-RBM can overcome the above sample confusing problem efficiently and effectively. To cope with the real-valued input samples, we further extend the binary RBM to Gaussian-Bernoulli RBM (GBRBM), leading to the CS-GBRBM (CM-GBRBM) feature extracting framework. Experiments on binary datasets, i.e., MNIST and USPS, scene image dataset (Scene-15), and object image dataset (Coil-100), well verify the above facts and show the competitive results.