Determining the Accuracy of Crowdsourced Tweet Verification for Auroral Research

The Aurorasaurus citizen science project harnesses volunteer crowdsourcing to identify sightings of an aurora (or the "northern/southern lights") posted by citizen scientists on Twitter. Previous studies have demonstrated that aurora sightings can be mined from Twitter but with the caveat that there is a high level of accompanying non-sighting tweets, especially during periods of low auroral activity. Aurorasaurus attempts to mitigate this, and thus increase the quality of its Twitter sighting data, by utilizing volunteers to sift through a pre-filtered list of geo-located tweets to verify real-time aurora sightings. In this study, the current implementation of this crowdsourced verification system, including the process of geo-locating tweets, is described and its accuracy (which, overall, is found to be 68.4%) is determined. The findings suggest that citizen science volunteers are able to accurately filter out unrelated, spam-like, Twitter data but struggle when filtering out somewhat related, yet undesired, data. The citizen scientists particularly struggle with determining the real-time nature of the sightings and care must therefore be taken when relying on crowdsourced identification