Open AccessDEEP NEURAL NETWORKS FOR SOCIAL VISUALS: STUDYING CLIMATE COMMUNICATION ON YOUTUBE
Author(s) -
Matteo Magnani,
Alexandra Segerberg
Publication year - 2021
Publication title -
selected papers of internet research
Language(s) - English
Resource type - Journals
ISSN - 2162-3317
DOI - 10.5210/spir.v2021i0.12204
Subject(s) - context (archaeology) , data science , social media , computer science , argument (complex analysis) , salient , interdependence , process (computing) , narrative , social network (sociolinguistics) , artificial intelligence , sociology , world wide web , social science , geography , art , biochemistry , chemistry , literature , archaeology , operating system
Visual politics is becoming increasingly salient online. Thequalitative methods of the research tradition do not expand to complex media ecologies, butadvances in deep neural networks open an unprecedented path to large-scale analysis on thebasis of actual visual content. However, the analysis of social visuals is challenging,since social and political scenes are semantically rich and convey complex narratives andideas. This paper examines validity conditions for integrating deep neural network tools inthe study of digitally augmented social visuals. It argues that the complexity of socialvisuals needs to be reflected in the validation process and its communication: It isnecessary to move beyond the conventionally dichotomous approach to neural networkvalidation which focuses on data and neural network respectively, to instead acknowledge theinterdependency between data and tool. The final definition of good data is not availableuntil the end of the process, which itself relies on a tool that needs good data to betrained. Themes change during the process not just because of our interaction with the data,but also because of our interactions with the tool and the specific way in which it mediatesour analysis. An upshot is that the conventional approach of performance assessment – i.e.,counting errors – is potentially misleading in this context. We explore our argumentexperimentally in the context of a study that addresses climate communication on YouTube.Climate themes such as polar bear in arctic landscapes and elite people/events present toughcases of social visuals.