Premium
Prospective validation study of an epilepsy seizure risk system for outpatient evaluation
Author(s) -
Chiang Sharon,
Goldenholz Daniel M.,
Moss Robert,
Rao Vikram R.,
Haneef Zulfi,
Theodore William H.,
Kleen Jonathan K.,
Gavvala Jay,
Vannucci Marina,
Stern John M.
Publication year - 2020
Publication title -
epilepsia
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.687
H-Index - 191
eISSN - 1528-1167
pISSN - 0013-9580
DOI - 10.1111/epi.16397
Subject(s) - epilepsy , inter rater reliability , intra rater reliability , medicine , seizure types , psychology , psychiatry , rating scale , developmental psychology , confidence interval
Objective We conducted clinical testing of an automated Bayesian machine learning algorithm (Epilepsy Seizure Assessment Tool [EpiSAT]) for outpatient seizure risk assessment using seizure counting data, and validated performance against specialized epilepsy clinician experts. Methods We conducted a prospective longitudinal study of EpiSAT performance against 24 specialized clinician experts at three tertiary referral epilepsy centers in the United States. Accuracy, interrater reliability, and intra‐rater reliability of EpiSAT for correctly identifying changes in seizure risk (improvements, worsening, or no change) were evaluated using 120 seizures from four synthetic seizure diaries (seizure risk known) and 120 seizures from four real seizure diaries (seizure risk unknown). The proportion of observed agreement between EpiSAT and clinicians was evaluated to assess compatibility of EpiSAT with clinical decision patterns by epilepsy experts. Results EpiSAT exhibited substantial observed agreement (75.4%) with clinicians for assessing seizure risk. The mean accuracy of epilepsy providers for correctly assessing seizure risk was 74.7%. EpiSAT accurately identified seizure risk in 87.5% of seizure diary entries, corresponding to a significant improvement of 17.4% ( P = .002). Clinicians exhibited low‐to‐moderate interrater reliability for seizure risk assessment (Krippendorff's α = 0.46) with good intrarater reliability across a 4‐ to 12‐week evaluation period (Scott's π = 0.89). Significance These results validate the ability of EpiSAT to yield objective clinical recommendations on seizure risk which follow decision patterns similar to those from specialized epilepsy providers, but with improved accuracy and reproducibility. This algorithm may serve as a useful clinical decision support system for quantitative analysis of clinical seizure frequency in clinical epilepsy practice.