Squeeze-SwinFormer: Spectral Squeeze and Excitation Swin Transformer Network for Hyperspectral Image Classification

Hyperspectral Images (HSIs) are highly complex, containing a richer spectral dimension compared to conventional images. Deep learning methods are increasingly being applied to process this three-dimensional data for Hyperspectral Image Classification (HSIC). The Vision Transformer model is steadily gaining prominence in computer vision, emerging as a potential alternative to traditional convolutional neural network (CNN) architectures. Although transformers are powerful due to their self-attention mechanisms, they face challenges related to scalability and efficiency, particularly when processing high-resolution images. In order to solve these problems, in this paper, we propose a novel Squeeze-SwinFormer network for HSIC. This novel approach integrates Spectral Squeeze and Excitation (SE) blocks into the Swin Transformer architecture to enhance feature extraction and attention mechanisms in the model. The SE block dynamically recalibrates channel-wise feature responses, improving the model's focus on significant features. The Squeeze-SwinFormer, equipped with window-based multi-head self-attention, efficiently captures long-range dependencies with reduced computational complexity by partitioning input data into non-overlapping windows. Additionally, the SE block's integration within the Swin Transformer enables improved global contextual understanding by selectively weighting the feature maps. We further refine the architecture using various components, such as patch extraction and embedding layers, and a patch merging strategy, ensuring efficient multi-scale feature extraction. The proposed architecture demonstrates superior performance in tasks requiring attention to intricate details in HSIC. The comprehensive experimental results demonstrate that our proposed model achieved higher test accuracies against the state-of-the-art method, with results of 99.93%, 98.31%, 99.50%, and 97.75% on the SA, IP, PU, and KSC public benchmark HSIC datasets, respectively. Our approach also shows good generalization ability when applied to new datasets. Overall, our proposed approach represents a promising direction for HSIC.