English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we show how to use audio to supervise the learning of activespeaker detection in video. Voice Activity Detection (VAD) guides the learningof the vision-based classifier in a weakly supervised manner. The classifieruses spatio-temporal features to encode upper body motion - facial expressionsand gesticulations associated with speaking. We further improve a generic modelfor active speaker detection by learning person specific models. Finally, wedemonstrate the online adaptation of generic models learnt on one dataset, topreviously unseen people in a new dataset, again using audio (VAD) for weaksupervision. The use of temporal continuity overcomes the lack of cleantraining data. We are the first to present an active speaker detection systemthat learns on one audio-visual dataset and automatically adapts to speakers ina new dataset. This work can be seen as an example of how the availability ofmulti-modal data allows us to learn a model without the need for supervision,by transferring knowledge from one modality to another.

Cross-modal Supervision for Learning Active Speaker Detection in Video