WinMRSI: Feature Matching With Window Attention for Multi-Modal Remote Sensing Image

Multi-modal remote sensing image matching is a crucial task with broad application potential. However, substantial nonlinear radiometric differences between multi-modal images pose significant challenges, often leading to mismatches. To tackle these challenges, this paper introduces WinMRSI, a window attention-based multi-modal remote sensing image matching method designed to enhance cross-modal feature extraction and information interaction. For feature extraction, a siamese network with Discrete Cosine Transform (DCT) is employed to model inter-channel dependencies and extract multi-scale features from cross-modal images. Additionally, a dual-branch network is designed to capture contextual dependencies while refining local feature representations. For information interaction, WinMRSI integrates a window attention mechanism to strengthen fine-grained feature fusion within highly relevant windows, enabling the model to focus on discriminative regions. Furthermore, a multi-level matching module progressively refines matching accuracy in a coarse-to-fine manner across window, patch, and pixel levels. Extensive evaluations on benchmark datasets demonstrate that WinMRSI achieves state-of-the-art performance in multi-modal remote sensing image matching. Ablation studies further validate the effectiveness of each component in WinMRSI.