English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In this paper, we propose a method to learn unified representations ofmultilingual speech and text with a single model, especially focusing on thepurpose of speech synthesis. We represent multilingual speech audio with speechunits, the quantized representations of speech features encoded from aself-supervised speech model. Therefore, we can focus on their linguisticcontent by treating the audio as pseudo text and can build a unifiedrepresentation of speech and text. Then, we propose to train an encoder-decoderstructured model with a Unit-to-Unit Translation (UTUT) objective onmultilingual data. Specifically, by conditioning the encoder with the sourcelanguage token and the decoder with the target language token, the model isoptimized to translate the spoken language into that of the target language, ina many-to-many language translation setting. Therefore, the model can build theknowledge of how spoken languages are comprehended and how to relate them todifferent languages. A single pre-trained model with UTUT can be employed fordiverse multilingual speech- and text-related tasks, such as Speech-to-SpeechTranslation (STS), multilingual Text-to-Speech Synthesis (TTS), andText-to-Speech Translation (TTST). By conducting comprehensive experimentsencompassing various languages, we validate the efficacy of the proposed methodacross diverse multilingual tasks. Moreover, we show UTUT can performmany-to-many language STS, which has not been previously explored in theliterature. Samples are available on https://choijeongsoo.github.io/utut.

Many-to-Many Spoken Language Translation via Unified Speech and Text Representation Learning with Unit-to-Unit Translation