Open AccessImages2Poem in different contexts with Dual‐CharRNN
Author(s) -
Yan Jie,
Xie Yuxiang,
Luan Xidao
Publication year - 2022
Publication title -
caai transactions on intelligence technology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.613
H-Index - 15
ISSN - 2468-2322
DOI - 10.1049/cit2.12089
Subject(s) - readability , poetry , consistency (knowledge bases) , dual (grammatical number) , computer science , image (mathematics) , chinese poetry , focus (optics) , artificial intelligence , natural language processing , linguistics , philosophy , physics , optics , programming language
Image to caption has attracted extensive research attention recently. However, image to poetry, especially Chinese classical poetry, is much more challenging. Previous works mainly focus on generating coherent poetry without taking the contexts of poetry into account. In this paper, we propose an Images2Poem with the Dual‐CharRNN model which exploits images to generate Chinese classical poems in different contexts. Specifically, we first extract a few keywords representing elements from the given image based on multi‐label image classification. Then, these keywords are expanded to related ones with the planning‐based model. Finally, we employ Dual‐CharRNN to generate Chinese classical poetry in different contexts. A comprehensive evaluation of human judgements demonstrates that our model achieves promising performance and is effective in enhancing poetry's semantic consistency, readability, and aesthetics. We present an Images2Poem with the Dual‐CharRNN model exploiting images to generate Chinese classical poems in different contexts, which effectively improves the semantic consistency, readability and aesthetics of the generated poetry.