Premium
Is machine learning prediction of Aβ positivity consistent? An assessment of multiple datasets
Author(s) -
Grecchi Elisabetta,
Grisan Enrico,
Buckley Christopher,
Wolber Jan
Publication year - 2020
Publication title -
alzheimer's and dementia
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 6.713
H-Index - 118
eISSN - 1552-5279
pISSN - 1552-5260
DOI - 10.1002/alz.040990
Subject(s) - artificial intelligence , machine learning , cross validation , lasso (programming language) , computer science , sensitivity (control systems) , algorithm , electronic engineering , world wide web , engineering
Background Recent applications of machine learning methodologies in AD suggest that synergistic utilisation of imaging and non‐imaging biomarkers may improve the ability to predict subject’s likelihood to present with amyloid pathology prior to performing a PET scan. In this work we developed an algorithm for prediction of patient amyloid positivity that shows robust performance across different databases, thus being more likely to be suitable for real‐world application. Method Machine learning (ML) algorithms that combine imaging (MRI volumes), genetic (ApoE status), psychometric (MMSE and CDR), and demographic (age/gender) data were developed to predict a patient’s probability of being Aβ positive. The patient’s [ 18 F]flutemetamol PET scan served as the standard of truth (SoT). Two ML methodologies (LASSO and RUS‐BOOST) were tested to tackle both the unbalance between Aβ positive/negative patients and selection of volumetric brain regions. The algorithms were trained and tested using combination of 5 different databases, excluding all healthy controls: MCI Progression Phase III trial (204 MCI), AIBL (52 MCI, 16 pAD), Biofinder (117 SCD, 147 AD), and a subset of ADNI data selected for the ADNI evolution prediction data (564 MCI, 147 AD). Result The cross‐validation results and the performance of the algorithms with different test sets are reported in Tables 1 and 2. We report AUC, accuracy, sensitivity, specificity, PPV and NPV for a probability threshold of 0.5 (Table1). PPV and NPV are also reported for threshold optimized and selected by the algorithm to provide a sensitivity of approximately 0.75 (Table 2), which results in a lower rate of misclassified amyloid positive. Conclusion We show that the developed algorithms can be confidently used across several independently acquired datasets. Our results suggest that for optimal usage as screening tool in a specific clinical trial, ML techniques should be adjusted for the characteristic of the specific population under analysis. Despite fairly consistent performance of the two methods within cross validation, results can vary when applying the learned models to different datasets, suggesting that cohort selection criteria, composition, and geographical origin may additionally influence outcomes.