English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Real-Time Magnetic resonance imaging (rtMRI) of the midsagittal plane of themouth is of interest for speech production research. In this work, we focus onestimating utterance level rtMRI video from the spoken phoneme sequence. Weobtain time-aligned phonemes from forced alignment, to obtain frame-levelphoneme sequences which are aligned with rtMRI frames. We propose asequence-to-sequence learning model with a transformer phoneme encoder andconvolutional frame decoder. We then modify the learning by using intermediaryfeatures obtained from sampling from a pretrained phoneme-conditionedvariational autoencoder (CVAE). We train on 8 subjects in a subject-specificmanner and demonstrate the performance with a subjective test. We also use anauxiliary task of air tissue boundary (ATB) segmentation to obtain theobjective scores on the proposed models. We show that the proposed method isable to generate realistic rtMRI video for unseen utterances, and adding CVAEis beneficial for learning the sequence-to-sequence mapping for subjects wherethe mapping is hard to learn.

Real-Time MRI Video synthesis from time aligned phonemes with sequence-to-sequence networks