English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Meetings are a common activity in professional contexts, and it remainschallenging to endow vocal assistants with advanced functionalities tofacilitate meeting management. In this context, a task like active speakerdetection can provide useful insights to model interaction between meetingparticipants. Motivated by our application context related to advanced meetingassistant, we want to combine audio and visual information to achieve the bestpossible performance. In this paper, we propose two different types of fusionfor the detection of the active speaker, combining two visual modalities and anaudio modality through neural networks. For comparison purpose, classicalunsupervised approaches for audio feature extraction are also used. We expectvisual data centered on the face of each participant to be very appropriate fordetecting voice activity, based on the detection of lip and facial gestures.Thus, our baseline system uses visual data and we chose a 3D ConvolutionalNeural Network architecture, which is effective for simultaneously encodingappearance and movement. To improve this system, we supplemented the visualinformation by processing the audio stream with a CNN or an unsupervisedspeaker diarization system. We have further improved this system by addingvisual modality information using motion through optical flow. We evaluated ourproposal with a public and state-of-the-art benchmark: the AMI corpus. Weanalysed the contribution of each system to the merger carried out in order todetermine if a given participant is currently speaking. We also discussed theresults we obtained. Besides, we have shown that, for our application context,adding motion information greatly improves performance. Finally, we have shownthat attention-based fusion improves performance while reducing the standarddeviation.

Audio-video fusion strategies for active speaker detection in meetings