A New Linear Classifier Based on Combining Supervised and Unsupervised Techniques

The aim of the research reported in the paper is to obtain an alternative approach in using Support Vector Machine (SVM) in case of non- linearly separable data based on using the k-means algorithm instead of the standard kernel based approach. The SVM is a relatively new concept in machine learning and it was introduced by Vapnik in 1995. In designing a classifier, two main problems have to be solved, on one hand the option concerning a suitable structure and on the other hand the selection of an algorithm for parameter estimation. The algorithm for parameter estimation performs the optimization of a conven- able selected cost function with respect to the empirical risk which is directly related to the representativeness of the available learning sequence. The choice of the structure is made such that to maximize the generalization capacity, that is to assure good performance in classifying new data coming from the same classes. In solving these problems one has to establish a balance between the accuracy in encoding the learning sequence and the generalization capacities because usually the over-fitting prevents the minimization of the empirical risk.