English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Training perception systems for self-driving cars requires substantialannotations. However, manual labeling in 2D images is highly labor-intensive.While existing datasets provide rich annotations for pre-recorded sequences,they fall short in labeling rarely encountered viewpoints, potentiallyhampering the generalization ability for perception models. In this paper, wepresent PanopticNeRF-360, a novel approach that combines coarse 3D annotationswith noisy 2D semantic cues to generate consistent panoptic labels andhigh-quality images from any viewpoint. Our key insight lies in exploiting thecomplementarity of 3D and 2D priors to mutually enhance geometry and semantics.Specifically, we propose to leverage noisy semantic and instance labels in both3D and 2D spaces to guide geometry optimization. Simultaneously, the improvedgeometry assists in filtering noise present in the 3D and 2D annotations bymerging them in 3D space via a learned semantic field. To further enhanceappearance, we combine MLP and hash grids to yield hybrid scene features,striking a balance between high-frequency appearance and predominantlycontiguous semantics. Our experiments demonstrate PanopticNeRF-360'sstate-of-the-art performance over existing label transfer methods on thechallenging urban scenes of the KITTI-360 dataset. Moreover, PanopticNeRF-360enables omnidirectional rendering of high-fidelity, multi-view andspatiotemporally consistent appearance, semantic and instance labels. We makeour code and data available at https://github.com/fuxiao0719/PanopticNeRF

PanopticNeRF-360: Panoramic 3D-to-2D Label Transfer in Urban Scenes