EyeNet++: A Multi-Scale and Multi-Density Approach for Outdoor 3D Semantic Segmentation Inspired by the Human Visual Field

In processing dense 3D point cloud data for semantic segmentation, current deep learning networks face challenges maintaining a sufficient neighbourhood search area, limiting their accuracy and scalability. Inspired by the human visual field, EyeNet++ introduces a novel multi-scale and multi-density input strategy, ensuring extensive spatial coverage of a neighbourhood search area while preserving detail. The network is designed to maximize feature extraction from these diverse inputs through innovative modules. Parallel feature processing, facilitated by Messenger Fusion Block, enables the efficient exchange of complementary features between input scales. The Dense Local Feature Aggregation Block enhances feature learning by expanding receptive fields and capturing local and global context. At the same time, the Feature Merge Block seamlessly integrates features from multiple scales and densities for comprehensive and precise representation. Validated on large-scale benchmark datasets such as SensatUrban, Toronto3D, DALES, YUTOSemantic, and Semantic3D, EyeNet++ achieves state-of-the-art performance on SensatUrban, Toronto3D, and YUTOSemantic, effectively addressing the challenges posed by dense point clouds in outdoor scenes with various object sizes. The source code is available at https://github.com/Yacovitch/EyeNet2.