Open AccessSwarm: A federated cloud framework for large-scale variant analysis
Author(s) -
Amir Bahmani,
Kyle Ferriter,
Vandhana Krishnan,
Arash Alavi,
Amir Alavi,
Philip S. Tsao,
M Snyder,
Cuiping Pan
Publication year - 2021
Publication title -
plos computational biology/plos computational biology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.628
H-Index - 182
eISSN - 1553-7358
pISSN - 1553-734X
DOI - 10.1371/journal.pcbi.1008977
Subject(s) - cloud computing , computer science , swarm behaviour , computation , crosstalk , distributed computing , database , data mining , computational biology , biology , operating system , artificial intelligence , engineering , algorithm , electronic engineering
Genomic data analysis across multiple cloud platforms is an ongoing challenge, especially when large amounts of data are involved. Here, we present Swarm, a framework for federated computation that promotes minimal data motion and facilitates crosstalk between genomic datasets stored on various cloud platforms. We demonstrate its utility via common inquiries of genomic variants across BigQuery in the Google Cloud Platform (GCP), Athena in the Amazon Web Services (AWS), Apache Presto and MySQL. Compared to single-cloud platforms, the Swarm framework significantly reduced computational costs, run-time delays and risks of security breach and privacy violation.