English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We investigate the benefit of combining blind audio recordings with 3D sceneinformation for novel-view acoustic synthesis. Given audio recordings from 2-4microphones and the 3D geometry and material of a scene containing multipleunknown sound sources, we estimate the sound anywhere in the scene. We identifythe main challenges of novel-view acoustic synthesis as sound sourcelocalization, separation, and dereverberation. While naively training anend-to-end network fails to produce high-quality results, we show thatincorporating room impulse responses (RIRs) derived from 3D reconstructed roomsenables the same network to jointly tackle these tasks. Our method outperformsexisting methods designed for the individual tasks, demonstrating itseffectiveness at utilizing 3D visual information. In a simulated study on theMatterport3D-NVAS dataset, our model achieves near-perfect accuracy on sourcelocalization, a PSNR of 26.44 dB and a SDR of 14.23 dB for source separationand dereverberation, resulting in a PSNR of 25.55 dB and a SDR of 14.20 dB onnovel-view acoustic synthesis. Code, pretrained model, and video results areavailable on the project webpage (https://github.com/apple/ml-nvas3d).

Novel-View Acoustic Synthesis from 3D Reconstructed Rooms