Facial Expression Recognition by Multi‐Scale Local Binary Patterns ( MLBP ) and Convolutional Neural Network ( CNN ) Features

ABSTRACT The quality of human‐computer interactions (HCI) has increased recently because of developments in artificial intelligence (AI) and machine learning methods, but there are still numerous obstacles to overcome. One of these difficulties that has been taken into account by several academics in recent years is the recognition of emotions via the processing of facial pictures. Most of the previously suggested solutions have drawbacks like poor accuracy and restrictions on the amount of emotions detected. On the other hand, researchers need to focus more on identifying the ideal feature set that results in maximum detection accuracy. This work addresses these issues by outlining a novel method for extracting the best face characteristics and their improved categorisation. Pre‐processing, feature extraction, feature selection and classification are the four phases of the suggested technique. Image normalisation and face recognition are steps in the pre‐processing stage. The ideal features are chosen using a black hole optimisation approach in the proposed method, which combines a Convolutional Neural Network (CNN) and Multi‐scale Local Binary Patterns (MLBP) to extract the feature. The next step is to categorise certain characteristics and identify facial emotions in the photos using Error Correcting Output Codes (ECOC). To lessen the issue's complexity, the suggested ECOC model combines a number of Support Vector Machine (SVM) classifiers. Results reveal that the proposed model has average accuracies of 98.9% and 79.82%, respectively, for the Yale and FER‐2013 datasets in recognising facial expressions, which shows an increase of at least 1% over the prior approaches.