BiLSTM-based Parallel CNN Models with Attention and Ensemble Mechanism for Twitter Sentiment Analysis

Due to the increased use of social networks, word-of-mouth analysis has become an effective tool in market research since firms can determine the users’ attitudes toward their brands. However, the current approaches to analyzing word-of-mouth on social media face significant challenges like lack of feature integration, low sentiment analysis efficiency, low topic identification efficiency, issues with data scarcity, and the high costs of annotations. These problems limit the ability to offer detailed and comprehensive analyses of user sentiment. Sentiment analysis has been found to have positive results when using deep learning. When used together, models like the Convolutional Neural Networks (CNN) and LSTM networks have significant high-performance results for text feature extraction and semantic relationship of the word. This article proposes a new model based on attention mechanisms and ensemble learning. Our model uses Google’s pre-trained Word2Vec embeddings to represent text as dense vectors. Three parallel CNN layers extract complex low-level features from embeddings after the embedding layer. Batch normalization and dropout are implemented to prevent overfitting. A bidirectional LSTM is incorporated on the CNN layers to capture semantic relationships between words. An attention mechanism is implemented to prioritize words that convey substantial sentiment information. Our methodology incorporates four classifiers to produce text class predictions. Among them, five algorithms are selected for evaluation: Ridge Classifier (RC), Linear Discriminant Analysis (LDA), Extra Trees (ET), and Light Gradient Boosting Machine (LightGBM). In addition, we set up the BiLSTM-based CNN model as our basic model and added it to this ensemble to enhance the model’s performance. To evaluate the validity of the developed model, experiments were performed on two sentiment-labeled datasets commonly used, and we compared it to some of the most efficient baseline models in the field of sentiment analysis and achieved better results.