An Enhanced Density Peak Clustering Algorithm with Dimensionality Reduction and Relative Density Normalization for High-Dimensional Duplicate Data

Cluster analysis is a fundamental method for studying big data problems, as it groups samples based on shared features. In cluster analysis, a particular class of big data problems is defined by large sample sizes, high dimensionality, and a substantial presence of duplicate samples. Traditional clustering algorithms often fail to adequately address such challenges, particularly in detecting low-density clusters. To tackle these challenges, this paper presents an enhanced variant of the Density Peaks clustering(DPC) algorithm, referred to as the Dimensionality Reduction and Relative Density Normalization Density Peaks Clustering(DRDN-DPC) algorithm. The DRDN-DPC algorithm incorporates sample count as a weighting factor during the clustering process, thereby refining the density estimation. Additionally, dimensionality reduction techniques are employed to alleviate the adverse effects of high-dimensional data. Moreover, relative density normalization is introduced to improve the detection of low-density clusters, thereby enhancing the overall clustering performance. Following a comprehensive analysis of the computational complexity in both temporal and spatial aspects, the DRDN-DPC is systematically evaluated using 12 standard benchmark datasets and high-dimensional benchmarks with added noise. Its performance is rigorously compared to several clustering algorithms, including the K-means, the DBSCAN, the original DPC algorithm and two modified version DPC, which are KNN-DPC and SNN-DPC. Furthermore, the DRDN-DPC is applied to real-world clustering problems, including molecular clustering in molecular dynamics (MD) simulations and clustering of Electronic Health Records (EHR). The results substantiate the practical efficacy of DRDN-DPC in both domains, highlighting its capability to address complex clustering tasks in diverse application contexts.