Premium
Semantic Reconstruction: Reconstruction of Semantically Segmented 3D Meshes via Volumetric Semantic Fusion
Author(s) -
Jeon Junho,
Jung Jinwoong,
Kim Jungeon,
Lee Seungyong
Publication year - 2018
Publication title -
computer graphics forum
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.578
H-Index - 120
eISSN - 1467-8659
pISSN - 0167-7055
DOI - 10.1111/cgf.13544
Subject(s) - computer science , point cloud , artificial intelligence , segmentation , rgb color model , computer vision , semantics (computer science) , polygon mesh , 3d reconstruction , voxel , deep learning , pattern recognition (psychology) , computer graphics (images) , programming language
Semantic segmentation partitions a given image or 3D model of a scene into semantically meaning parts and assigns predetermined labels to the parts. With well‐established datasets, deep networks have been successfully used for semantic segmentation of RGB and RGB‐D images. On the other hand, due to the lack of annotated large‐scale 3D datasets, semantic segmentation for 3D scenes has not yet been much addressed with deep learning. In this paper, we present a novel framework for generating semantically segmented triangular meshes of reconstructed 3D indoor scenes using volumetric semantic fusion in the reconstruction process. Our method integrates the results of CNN‐based 2D semantic segmentation that is applied to the RGB‐D stream used for dense surface reconstruction. To reduce the artifacts from noise and uncertainty of single‐view semantic segmentation, we introduce adaptive integration for the volumetric semantic fusion and CRF‐based semantic label regularization. With these methods, our framework can easily generate a high‐quality triangular mesh of the reconstructed 3D scene with dense (i.e., per‐vertex) semantic labels. Extensive experiments demonstrate that our semantic segmentation results of 3D scenes achieves the state‐of‐the‐art performance compared to the previous voxel‐based and point cloud‐based methods.