Progressive Cluster-Guided Knowledge Distillation for Remote Sensing Image Scene Classification

Knowledge distillation has recently demonstrated remarkable potential in developing lightweight convolutional neural networks (CNNs) for remote sensing image (RSI) scene classification tasks. However, existing KD methods neglect the high inter-class similarity and significant intra-class variability, as well as the imbalance of data features in remote sensing images. To address these challenges, this paper proposes a novel and effective framework called progressive Clustering-Guided Knowledge Distillation (CGKD) for RSI scene classification. Specifically, we devise an Optimal Clustering Module that adaptively selects informative samples and dynamically updates cluster centers to alleviate intra-class and inter-class discrepancies. Subsequently, a Feature Selection Module is introduced to evaluate feature confidence levels, enabling smooth separation of features with varying difficulty and resolving feature imbalance. Furthermore, we design a dynamic learning loss that calculates adaptive weights to features based on their learning difficulty. By this manner, the teacher transfers more complete and higher-quality knowledge to the student. The experimental results on three benchmark datasets, including AID, UCMerced and NWPU-RESISC datasets, demonstrate that the proposed CGKD outperforms existing distillation methods.