Premium
Forecasting rare disease outbreaks from open source indicators
Author(s) -
Rekatsinas Theodoros,
Ghosh Saurav,
Mekaru Sumiko R.,
Nsoesie Elaine O.,
Brownstein John S.,
Getoor Lise,
Ramakrishnan Naren
Publication year - 2017
Publication title -
statistical analysis and data mining: the asa data science journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.381
H-Index - 33
eISSN - 1932-1872
pISSN - 1932-1864
DOI - 10.1002/sam.11337
Subject(s) - computer science , outbreak , multi source , data mining , data source , anomaly detection , open source , data science , statistics , mathematics , virology , biology , software , programming language
Background Rapidly increasing volumes of news feeds from diverse data sources, such as online newspapers, Twitter, and online blogs, are proving to be extremely valuable resources in helping to anticipate, detect, and forecast outbreaks of rare diseases. The goal of this paper is to develop techniques that can effectively forecast the emergence and progression of rare infectious diseases by combining data from disparate data sources. Methods We introduce SourceSeer, a novel algorithmic framework that combines spatiotemporal topic models with source‐based anomaly detection techniques. SourceSeer is capable of discovering the location focus of each source, allowing sources to be used as experts with varying degrees of authoritativeness. To fuse the individual source predictions into a final outbreak prediction, we employ a multiplicative weights algorithm taking into account the accuracy of each source. Results We evaluate the performance of SourceSeer using incidence data for hantavirus syndromes in multiple countries of Latin America provided by HealthMap over a time span of 15 months. We demonstrate that SourceSeer makes predictions of increased accuracy compared to several baselines and can forecast disease outbreaks in a timely manner even when no outbreaks were previously reported.