Open AccessNatural language description of images using hybrid recurrent neural network
Author(s) -
Md. Asifuzzaman Jishan,
Khan Raqib Mahmud,
Abul K. Azad
Publication year - 2019
Publication title -
international journal of electrical and computer engineering
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.277
H-Index - 22
ISSN - 2088-8708
DOI - 10.11591/ijece.v9i4.pp2932-2940
Subject(s) - computer science , recurrent neural network , artificial intelligence , benchmark (surveying) , convolutional neural network , natural language , natural language processing , line (geometry) , word (group theory) , representation (politics) , artificial neural network , pattern recognition (psychology) , object (grammar) , language model , linguistics , philosophy , geometry , mathematics , geodesy , politics , political science , law , geography
We presented a learning model that generated natural language description of images. The model utilized the connections between natural language and visual data by produced text line based contents from a given image. Our Hybrid Recurrent Neural Network model is based on the intricacies of Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Bi-directional Recurrent Neural Network (BRNN) models. We conducted experiments on three benchmark datasets, e.g., Flickr8K, Flickr30K, and MS COCO. Our hybrid model utilized LSTM model to encode text line or sentences independent of the object location and BRNN for word representation, this reduced the computational complexities without compromising the accuracy of the descriptor. The model produced better accuracy in retrieving natural language based description on the dataset.