Premium
Understanding the relevance between text and images in disaster‐related social media posts
Author(s) -
Mao Jin,
Zhang Ji,
Lu Kun,
Li Gang
Publication year - 2019
Publication title -
proceedings of the association for information science and technology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.193
H-Index - 14
ISSN - 2373-9231
DOI - 10.1002/pra2.151
Subject(s) - relevance (law) , similarity (geometry) , social media , computer science , information retrieval , literal (mathematical logic) , microblogging , annotation , image (mathematics) , artificial intelligence , world wide web , algorithm , political science , law
Disaster‐related social media posts are often attached with images, either relevant or irrelevant to the text. The relevance between text and attached images can be indicative of the quality of posts because relevant images bring enriched information beyond text. A relevance framework is proposed to depict four aspects of relevance between text and images. A dataset about Typhoon Mangkhut was collected from a microblog application in China. A random sample of 3,000 posts along with 6,217 images were independently annotated. In addition, an automated text and image similarity measure is introduced. Annotation results show that more images are relevant to text on the aspects of object and scene relevance than on literal and behavior relevance. Irrelevant images account for roughly 10% of all images. Text and image pairs with different relevance aspects show different similarity scores. The goal of this study is to improve our understanding of the relationship between text and images in disaster‐related social media posts, which have implications for applications based on text and images integration.