English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Applying NeRF to downstream perception tasks for scene understanding andrepresentation is becoming increasingly popular. Most existing methods treatsemantic prediction as an additional rendering task, \textit{i.e.}, the "labelrendering" task, to build semantic NeRFs. However, by renderingsemantic/instance labels per pixel without considering the contextualinformation of the rendered image, these methods usually suffer from unclearboundary segmentation and abnormal segmentation of pixels within an object. Tosolve this problem, we propose Generalized Perception NeRF (GP-NeRF), a novelpipeline that makes the widely used segmentation model and NeRF work compatiblyunder a unified framework, for facilitating context-aware 3D scene perception.To accomplish this goal, we introduce transformers to aggregate radiance aswell as semantic embedding fields jointly for novel views and facilitate thejoint volumetric rendering of both fields. In addition, we propose twoself-distillation mechanisms, i.e., the Semantic Distill Loss and theDepth-Guided Semantic Distill Loss, to enhance the discrimination and qualityof the semantic field and the maintenance of geometric consistency. Inevaluation, we conduct experimental comparisons under two perception tasks(\textit{i.e.} semantic and instance segmentation) using both synthetic andreal-world datasets. Notably, our method outperforms SOTA approaches by 6.94\%,11.76\%, and 8.47\% on generalized semantic segmentation, finetuning semanticsegmentation, and instance segmentation, respectively.

GP-NeRF: Generalized Perception NeRF for Context-Aware 3D Scene Understanding