Open AccessAutomatic Image Captioning Using Neural Networks
Author(s) -
Subash Pandey,
Rabin Kumar Dhamala,
Bikram Karki,
Sumit Dahal,
Rama Bastola
Publication year - 2020
Publication title -
journal of innovations in engineering education
Language(s) - English
Resource type - Journals
eISSN - 2773-823X
pISSN - 2594-343X
DOI - 10.3126/jiee.v3i1.34335
Subject(s) - closed captioning , computer science , artificial intelligence , convolutional neural network , image (mathematics) , encoder , word (group theory) , task (project management) , field (mathematics) , artificial neural network , natural language , computer vision , recurrent neural network , speech recognition , natural language processing , pattern recognition (psychology) , linguistics , philosophy , mathematics , management , pure mathematics , economics , operating system
Automatically generating a natural language description of an image is a major challenging task in the field of artificial intelligence. Generating description of an image bring together the fields: Natural Language Processing and Computer Vision. There are two types of approaches i.e. top-down and bottom-up. For this paper, we approached top-down that starts from the image and converts it into the word. Image is passed to Convolutional Neural Network (CNN) encoder and the output from it is fed further to Recurrent Neural Network (RNN) decoder that generates meaningful captions. We generated the image description by passing the real time images from the camera of a smartphone as well as tested with the test images from the dataset. To evaluate the model performance, we used BLEU (Bilingual Evaluation Understudy) score and match predicted words to the original caption.