English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Understanding the underlying relationship between tongue and oropharyngealmuscle deformation seen in tagged-MRI and intelligible speech plays animportant role in advancing speech motor control theories and treatment ofspeech related-disorders. Because of their heterogeneous representations,however, direct mapping between the two modalities -- i.e., two-dimensional(mid-sagittal slice) plus time tagged-MRI sequence and its correspondingone-dimensional waveform -- is not straightforward. Instead, we resort totwo-dimensional spectrograms as an intermediate representation, which containsboth pitch and resonance, from which to develop an end-to-end deep learningframework to translate from a sequence of tagged-MRI to its corresponding audiowaveform with limited dataset size.~Our framework is based on a novel fullyconvolutional asymmetry translator with guidance of a self residual attentionstrategy to specifically exploit the moving muscular structures duringspeech.~In addition, we leverage a pairwise correlation of the samples with thesame utterances with a latent space representation disentanglementstrategy.~Furthermore, we incorporate an adversarial training approach withgenerative adversarial networks to offer improved realism on our generatedspectrograms.~Our experimental results, carried out with a total of 63tagged-MRI sequences alongside speech acoustics, showed that our frameworkenabled the generation of clear audio waveforms from a sequence of tagged-MRI,surpassing competing methods. Thus, our framework provides the great potentialto help better understand the relationship between the two modalities.

Tagged-MRI Sequence to Audio Synthesis via Self Residual Attention Guided Heterogeneous Translator