Open AccessMultimodal Fusion for Image and Text Classification with Feature Selection and Dimension Reduction
Author(s) -
Xinran Liu,
Zhongju Wang,
Long Wang
Publication year - 2021
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1871/1/012064
Subject(s) - the internet , computer science , feature selection , dimension (graph theory) , feature (linguistics) , artificial intelligence , selection (genetic algorithm) , task (project management) , dimensionality reduction , reduction (mathematics) , relation (database) , information retrieval , machine learning , pattern recognition (psychology) , data mining , world wide web , mathematics , engineering , linguistics , philosophy , geometry , systems engineering , pure mathematics
Internet has become an important information platform, and it is very important to accurately understand the multimedia information of the Internet. In this paper, our main task is to do classification based on pictures and texts collected from the Internet, which is a classification problem of multimodal fusion in practice. However, when multimodal data is put together, there may occur the dimension disaster problem. We apply feature selection (FS) and dimension reduction (DR) in feature levels both in later fusion and early fusion to solve this problem. The classification accuracies in different models obtain improvements in different levels respectively. We also discuss the relation between single modals and multimodal in later fusion. In our experiments, images and text can be classified by multimodal models under FS/DR, and of which with the help the multimedia information from the Internet can be analysed better to help enterprises provide better services and products, and then carry out better network marketing and promotion.