English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Understanding and controlling latent representations in deep generativemodels is a challenging yet important problem for analyzing, transforming andgenerating various types of data. In speech processing, inspiring from theanatomical mechanisms of phonation, the source-filter model considers thatspeech signals are produced from a few independent and physically meaningfulcontinuous latent factors, among which the fundamental frequency $f_0$ and theformants are of primary importance. In this work, we start from a variationalautoencoder (VAE) trained in an unsupervised manner on a large dataset ofunlabeled natural speech signals, and we show that the source-filter model ofspeech production naturally arises as orthogonal subspaces of the VAE latentspace. Using only a few seconds of labeled speech signals generated with anartificial speech synthesizer, we propose a method to identify the latentsubspaces encoding $f_0$ and the first three formant frequencies, we show thatthese subspaces are orthogonal, and based on this orthogonality, we develop amethod to accurately and independently control the source-filter speech factorswithin the latent subspaces. Without requiring additional information such astext or human-labeled data, this results in a deep generative model of speechspectrograms that is conditioned on $f_0$ and the formant frequencies, andwhich is applied to the transformation speech signals. Finally, we also proposea robust $f_0$ estimation method that exploits the projection of a speechsignal onto the learned latent subspace associated with $f_0$.

Learning and controlling the source-filter representation of speech with a variational autoencoder