Open AccessChallenges in benchmarking metagenomic profilers
Author(s) -
Zheng Sun,
Shi Huang,
Meng Zhang,
Qiyun Zhu,
Niina Haiminen,
Anna Paola Carrieri,
Yoshiki Vázquez-Baeza,
Laxmi Parida,
Ho Cheol Kim,
Rob Knight,
Yang Yu Liu
Publication year - 2021
Publication title -
nature methods
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 19.469
H-Index - 318
eISSN - 1548-7105
pISSN - 1548-7091
DOI - 10.1038/s41592-021-01141-3
Subject(s) - metagenomics , benchmarking , computational biology , computer science , data science , biology , business , genetics , gene , marketing
Accurate microbial identification and abundance estimation are crucial for metagenomics analysis. Various methods for classification of metagenomic data and estimation of taxonomic profiles, broadly referred to as metagenomic profilers, have been developed. Nevertheless, benchmarking of metagenomic profilers remains challenging because some tools are designed to report relative sequence abundance while others report relative taxonomic abundance. Here we show how misleading conclusions can be drawn by neglecting this distinction between relative abundance types when benchmarking metagenomic profilers. Moreover, we show compelling evidence that interchanging sequence abundance and taxonomic abundance will influence both per-sample summary statistics and cross-sample comparisons. We suggest that the microbiome research community pay attention to potentially misleading biological conclusions arising from this issue when benchmarking metagenomic profilers, by carefully considering the type of abundance data that were analyzed and interpreted and clearly stating the strategy used for metagenomic profiling.