English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recently, the RGB images and point clouds fusion methods have been proposedto jointly estimate 2D optical flow and 3D scene flow. However, as bothconventional RGB cameras and LiDAR sensors adopt a frame-based data acquisitionmechanism, their performance is limited by the fixed low sampling rates,especially in highly-dynamic scenes. By contrast, the event camera canasynchronously capture the intensity changes with a very high temporalresolution, providing complementary dynamic information of the observed scenes.In this paper, we incorporate RGB images, Point clouds and Events for jointoptical flow and scene flow estimation with our proposed multi-stage multimodalfusion model, RPEFlow. First, we present an attention fusion module with across-attention mechanism to implicitly explore the internal cross-modalcorrelation for 2D and 3D branches, respectively. Second, we introduce a mutualinformation regularization term to explicitly model the complementaryinformation of three modalities for effective multimodal feature learning. Wealso contribute a new synthetic dataset to advocate further research.Experiments on both synthetic and real datasets show that our model outperformsthe existing state-of-the-art by a wide margin. Code and dataset is availableat https://npucvr.github.io/RPEFlow.

RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation