Open AccessFusing Near-Infrared Spectroscopy With Wearable Hemodynamic Measurements Improves Classification of Mental Stress
Author(s) -
Nil Z. Gurel,
Hewon Jung,
Sinan Hersek,
Omer T. Inan
Publication year - 2019
Publication title -
ieee sensors journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.681
H-Index - 121
eISSN - 1558-1748
pISSN - 1530-437X
DOI - 10.1109/jsen.2018.2872651
Subject(s) - wearable computer , mental arithmetic , recall , computer science , functional near infrared spectroscopy , artificial intelligence , photoplethysmogram , pattern recognition (psychology) , prefrontal cortex , speech recognition , heart rate , medicine , psychology , computer vision , cognition , blood pressure , neuroscience , embedded system , filter (signal processing) , cognitive psychology
Human-computer interaction (HCI) technology, and the automatic classification of a person's mental state, are of interest to multiple industries. In this work, the fusion of sensing modalities that monitor the oxygenation of the human prefrontal cortex (PFC) and cardiovascular physiology was evaluated to differentiate between rest, mental arithmetic and N-back memory tasks. A flexible headband to measure near-infrared spectroscopy (NIRS) for quantifying PFC oxygenation, and forehead photoplethysmography (PPG) for assessing peripheral cardiovascular activity was designed. Physiological signals such as the electrocardiogram (ECG) and seismocardiogram (SCG) were collected, along with the measurements obtained using the headband. The setup was tested and validated with a total of 16 human subjects performing a series of arithmetic and N-back memory tasks. Features extracted were related to cardiac and peripheral sympathetic activity, vasomotor tone, pulse wave propagation, and oxygenation. Machine learning techniques were utilized to classify rest, arithmetic, and N-back tasks, using leave-one-subject-out cross validation. Macro-averaged accuracy of 85%, precision of 84%, recall rate of 83%, and F1 score of 80% were obtained from the classification of the three states. Statistical analyses on the subject-based results demonstrate that the fusion of NIRS and peripheral cardiovascular sensing significantly improves the accuracy, precision, recall, and F1 scores, compared to using NIRS sensing alone. Moreover, the fusion significantly improves the precision compared to peripheral cardiovascular sensing alone. The results of this work can be used in the future to design a multi-modal wearable sensing system for classifying mental state for applications such as acute stress detection.