English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Turn-taking, aiming to decide when the next speaker can start talking, is anessential component in building human-robot spoken dialogue systems. Previousstudies indicate that multimodal cues can facilitate this challenging task.However, due to the paucity of public multimodal datasets, current methods aremostly limited to either utilizing unimodal features or simplistic multimodalensemble models. Besides, the inherent class imbalance in real scenario, e.g.sentence ending with short pause will be mostly regarded as the end of turn,also poses great challenge to the turn-taking decision. In this paper, we firstcollect a large-scale annotated corpus for turn-taking with over 5,000 realhuman-robot dialogues in speech and text modalities. Then, a novel gatedmultimodal fusion mechanism is devised to utilize various informationseamlessly for turn-taking prediction. More importantly, to tackle the dataimbalance issue, we design a simple yet effective data augmentation method toconstruct negative instances without supervision and apply contrastive learningto obtain better feature representations. Extensive experiments are conductedand the results demonstrate the superiority and competitiveness of our modelover several state-of-the-art baselines.

Gated Multimodal Fusion with Contrastive Learning for Turn-taking Prediction in Human-robot Dialogue