English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose an online 3D semantic segmentation method that incrementallyreconstructs a 3D semantic map from a stream of RGB-D frames. Unlike offlinemethods, ours is directly applicable to scenarios with real-time constraints,such as robotics or mixed reality. To overcome the inherent challenges ofonline methods, we make two main contributions. First, to effectively extractinformation from the input RGB-D video stream, we jointly estimate geometry andsemantic labels per frame in 3D. A key focus of our approach is to reason aboutsemantic entities both in the 2D input and the local 3D domain to leveragedifferences in spatial context and network architectures. Our method predicts2D features using an off-the-shelf segmentation network. The extracted 2Dfeatures are refined by a lightweight 3D network to enable reasoning about thelocal 3D structure. Second, to efficiently deal with an infinite stream ofinput RGB-D frames, a subsequent network serves as a temporal expert predictingthe incremental scene updates by leveraging 2D, 3D, and past information in alearned manner. These updates are then integrated into a global scenerepresentation. Using these main contributions, our method can enable scenarioswith real-time constraints and can scale to arbitrary scene sizes by processingand updating the scene only in a local region defined by the new measurement.Our experiments demonstrate improved results compared to existing onlinemethods that purely operate in local regions and show that complementarysources of information can boost the performance. We provide a thoroughablation study on the benefits of different architectural as well asalgorithmic design decisions. Our method yields competitive results on thepopular ScanNet benchmark and SceneNN dataset.

ALSTER: A Local Spatio-Temporal Expert for Online 3D Semantic Reconstruction