English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The tongue's intricate 3D structure, comprising localized functional units,plays a crucial role in the production of speech. When measured using taggedMRI, these functional units exhibit cohesive displacements and derivedquantities that facilitate the complex process of speech production.Non-negative matrix factorization-based approaches have been shown to estimatethe functional units through motion features, yielding a set of building blocksand a corresponding weighting map. Investigating the link between weightingmaps and speech acoustics can offer significant insights into the intricateprocess of speech production. To this end, in this work, we utilizetwo-dimensional spectrograms as a proxy representation, and develop anend-to-end deep learning framework for translating weighting maps to theircorresponding audio waveforms. Our proposed plastic light transformer (PLT)framework is based on directional product relative position bias andsingle-level spatial pyramid pooling, thus enabling flexible processing ofweighting maps with variable size to fixed-size spectrograms, without inputinformation loss or dimension expansion. Additionally, our PLT frameworkefficiently models the global correlation of wide matrix input. To improve therealism of our generated spectrograms with relatively limited training samples,we apply pair-wise utterance consistency with Maximum Mean Discrepancyconstraint and adversarial training. Experimental results on a dataset of 29subjects speaking two utterances demonstrated that our framework is able tosynthesize speech audio waveforms from weighting maps, outperformingconventional convolution and transformer models.

Speech Audio Synthesis from Tagged MRI and Non-Negative Matrix Factorization via Plastic Transformer