OS-CycleGAN: Modified CycleGAN-based Descriptors for Optical and SAR Image Matching

Due to the complementary nature of information contained within Synthetic Aperture Radar (SAR) and optical imagery modalities, accurate registration of these images is a crucial process in remote sensing applications. However, precise matching becomes a complex challenge as a consequence of the substantial radiometric and geometric discrepancies between the two modalities. While recent techniques based on Generative Adversarial Networks (GANs) have proposed viable solutions for SAR and optical image matching, there remains significant potential for further improvements. In this paper, we propose a modified CycleGAN algorithm (OS-CycleGAN) to perform optical and SAR image matching. The loss function of the OS-CycleGAN network is defined not only to enhance the adversarial and cycle consistency losses but also to converge the latent spaces of the generators. By doing so, the latent spaces can be utilized as feature vectors or descriptors. Consequently, in addition to image translation, OS-CycleGAN’s generators provide new descriptors for the central pixels of SAR and optical patches. The method also employs decision-making modules to render the descriptors invariant to projective transformations such as rotation and shearing, as well as contrast changes. Furthermore, by disregarding SAR keypoints in areas affected by geometric distortions such as layover, foreshortening, and radar shadow, and removing optical keypoints in areas affected by optical shadow, the proposed approach mitigates outliers and improves the Correct Matches Rate (CMR). The conducted experimental results on the Sentinel-1/2 and OpenEarthMap-SAR datasets demonstrate the efficacy of the suggested method in terms of matching accuracy and robustness.