English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper describes the practical response- and performance-awaredevelopment of online speech enhancement for an augmented reality (AR) headsetthat helps a user understand conversations made in real noisy echoicenvironments (e.g., cocktail party). One may use a state-of-the-art blindsource separation method called fast multichannel nonnegative matrixfactorization (FastMNMF) that works well in various environments thanks to itsunsupervised nature. Its heavy computational cost, however, prevents itsapplication to real-time processing. In contrast, a supervised beamformingmethod that uses a deep neural network (DNN) for estimating spatial informationof speech and noise readily fits real-time processing, but suffers from drasticperformance degradation in mismatched conditions. Given such complementarycharacteristics, we propose a dual-process robust online speech enhancementmethod based on DNN-based beamforming with FastMNMF-guided adaptation. FastMNMF(back end) is performed in a mini-batch style and the noisy and enhanced speechpairs are used together with the original parallel training data for updatingthe direction-aware DNN (front end) with backpropagation at acomputationally-allowable interval. This method is used with a blinddereverberation method called weighted prediction error (WPE) for transcribingthe noisy reverberant speech of a speaker, which can be detected from video orselected by a user's hand gesture or eye gaze, in a streaming manner andspatially showing the transcriptions with an AR technique. Our experimentshowed that the word error rate was improved by more than 10 points with therun-time adaptation using only twelve minutes of observation.

Direction-Aware Adaptive Online Neural Speech Enhancement with an Augmented Reality Headset in Real Noisy Conversational Environments