Open AccessAutomatic Chinese Meme Generation Using Deep Neural Networks
Author(s) -
Lin Wang,
Qimeng Zhang,
Youngbin Kim,
Ruizheng Wu,
Hongyu Jin,
Haoke Deng,
Pengchu Luo,
Chang-Hun Kim
Publication year - 2021
Publication title -
ieee access
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.587
H-Index - 127
ISSN - 2169-3536
DOI - 10.1109/access.2021.3127324
Subject(s) - aerospace , bioengineering , communication, networking and broadcast technologies , components, circuits, devices and systems , computing and processing , engineered materials, dielectrics and plasmas , engineering profession , fields, waves and electromagnetics , general topics for engineers , geoscience , nuclear engineering , photonics and electrooptics , power, energy and industry applications , robotics and control systems , signal processing and analysis , transportation
Internet memes have become widely used by people for online communication and interaction, particularly through social media. Interest in meme-generation research has been increasing rapidly. In this study, we address the problem of meme generation as an image captioning task, which uses an encoder–decoder architecture to generate Chinese meme texts that match image content. First, to train the model on the characteristics of Chinese memes, we collected a dataset of 3,000 meme images with 30,000 corresponding humorous Chinese meme texts. Second, we introduced a Chinese meme generation system that can generate humorous and relevant texts from any given image. Our system used a pre-trained ResNet-50 for image feature extraction and a state-of-the-art transformer-based GPT-2 model to generate Chinese meme texts. Finally, we combined the generated text and images to form common image memes. We performed qualitative evaluations of the generated Chinese meme texts through different user studies. The evaluation results revealed that the Chinese memes generated by our model were indistinguishable from real ones.