English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Depth estimation, visual odometry (VO), and bird's-eye-view (BEV) scenelayout estimation present three critical tasks for driving scene perception,which is fundamental for motion planning and navigation in autonomous driving.Though they are complementary to each other, prior works usually focus on eachindividual task and rarely deal with all three tasks together. A naive way isto accomplish them independently in a sequential or parallel manner, but thereare many drawbacks, i.e., 1) the depth and VO results suffer from the inherentscale ambiguity issue; 2) the BEV layout is directly predicted from thefront-view image without using any depth-related information, although thedepth map contains useful geometry clues for inferring scene layouts. In thispaper, we address these issues by proposing a novel joint perception frameworknamed JPerceiver, which can simultaneously estimate scale-aware depth and VO aswell as BEV layout from a monocular video sequence. It exploits the cross-viewgeometric transformation (CGT) to propagate the absolute scale from the roadlayout to depth and VO based on a carefully-designed scale loss. Meanwhile, across-view and cross-modal transfer (CCT) module is devised to leverage thedepth clues for reasoning road and vehicle layout through an attentionmechanism. JPerceiver can be trained in an end-to-end multi-task learning way,where the CGT scale loss and CCT module promote inter-task knowledge transferto benefit feature learning of each task. Experiments on Argoverse, Nuscenesand KITTI show the superiority of JPerceiver over existing methods on all theabove three tasks in terms of accuracy, model size, and inference speed. Thecode and models are availableat~\href{https://github.com/sunnyHelen/JPerceiver}{https://github.com/sunnyHelen/JPerceiver}.

JPerceiver: Joint Perception Network for Depth, Pose and Layout Estimation in Driving Scenes