Sign Language Recognition Using Multiple Kernel Learning: A Case Study of Pakistan Sign Language

All over the world, deaf people use sign language as the only reliable source of communication with each other as well as with normal people. These communicating signs are made up of the shape of the hand and movement. In Pakistan, deaf people use Pakistan sign language (PSL) as a means of communication with people. In scientific literature, many studies have been done on PSL recognition and classification. Most of these work focused on colored-based hands while some others are sensors and Kinect-based approaches. These techniques are costly and also avoid user-friendliness. In this paper, a technique is proposed for the recognition of thirty-six static alphabets of PSL using bare hands. The dataset is obtained from the sign language videos. At a later step, four vision-based features are extracted i.e. local binary patterns, a histogram of oriented gradients, edge-oriented histogram, and speeded up robust features. The extracted features are individually classified using Multiple kernel learning (MKL) in support vector machine (SVM). We employed a one-to-all approach for the implementation of basic binary SVM into the multi-class SVM. A voting scheme is adopted for the final recognition of PSL. The performance of the proposed technique is measured in terms of accuracy, precision, recall, and F-score. The simulation results are promising as compared with existing approaches.