English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we propose a novel multi-modal multi-task encoder-decoderpre-training framework (MMSpeech) for Mandarin automatic speech recognition(ASR), which employs both unlabeled speech and text data. The main difficultyin speech-text joint pre-training comes from the significant difference betweenspeech and text modalities, especially for Mandarin speech and text. UnlikeEnglish and other languages with an alphabetic writing system, Mandarin uses anideographic writing system where character and sound are not tightly mapped toone another. Therefore, we propose to introduce the phoneme modality intopre-training, which can help capture modality-invariant information betweenMandarin speech and text. Specifically, we employ a multi-task learningframework including five self-supervised and supervised tasks with speech andtext data. For end-to-end pre-training, we introduce self-supervisedspeech-to-pseudo-codes (S2C) and phoneme-to-text (P2T) tasks utilizingunlabeled speech and text data, where speech-pseudo-codes pairs andphoneme-text pairs are a supplement to the supervised speech-text pairs. Totrain the encoder to learn better speech representation, we introduceself-supervised masked speech prediction (MSP) and supervised phonemeprediction (PP) tasks to learn to map speech into phonemes. Besides, wedirectly add the downstream supervised speech-to-text (S2T) task into thepre-training process, which can further improve the pre-training performanceand achieve better recognition results even without fine-tuning. Experiments onAISHELL-1 show that our proposed method achieves state-of-the-art performance,with a more than 40% relative improvement compared with other pre-trainingmethods.

MMSpeech: Multi-modal Multi-task Encoder-Decoder Pre-training for Speech Recognition