Physiologically-Guided Cross-Modal Fusion for Robust Vigilance Estimation

Vigilance estimation is crucial in human-machine systems such as autonomous driving and industrial monitoring. Electroencephalography (EEG) and electrooculography (EOG), sensing brain activity and eye movements respectively, are widely employed in fatigue detection. However, many current methods that combine these signals ignore the causality that changes in brain activity occur first and then express in eye movements. We introduce an EEG-EOG cross-modal fusion framework (E 2 CF) to address this issue. This framework uses an asymmetric attention mechanism to model the cause-and-effect relationship between brain signals and eye movement signals. In this attention architec-ture, EEG acts as Query that guide feature extraction, while EOG serves as reliable references (Value and Key). The framework integrates a depthwise separable convolution (DSC) and LSTM module for EEG, and an outer product em-bedding module for EOG. Extensive evaluations on public dataset demonstrate state-of-the-art performance in both intra- and inter-subject experiments, with CORR 0.995/RMSE 0.023 in intra-session and CORR 0.915/RMSE 0.087 in inter-session. These results highlight the potential of the proposed framework to deliver interpretable, and robust fusion of multimodal sensing.