English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

End-to-end ASR models trained on large amount of data tend to be implicitlybiased towards language semantics of the training data. Internal language modelestimation (ILME) has been proposed to mitigate this bias for autoregressivemodels such as attention-based encoder-decoder and RNN-T. Typically, ILME isperformed by modularizing the acoustic and language components of the modelarchitecture, and eliminating the acoustic input to perform log-linearinterpolation with the text-only posterior. However, for CTC-based ASR, it isnot as straightforward to decouple the model into such acoustic and languagecomponents, as CTC log-posteriors are computed in a non-autoregressive manner.In this work, we propose a novel ILME technique for CTC-based ASR models. Ourmethod iteratively masks the audio timesteps to estimate a pseudolog-likelihood of the internal LM by accumulating log-posteriors for only themasked timesteps. Extensive evaluation across multiple out-of-domain datasetsreveals that the proposed approach improves WER by up to 9.8% and OOV F1-scoreby up to 24.6% relative to Shallow Fusion, when only text data from targetdomain is available. In the case of zero-shot domain adaptation, with no accessto any target domain data, we demonstrate that removing the source domain biaswith ILME can still outperform Shallow Fusion to improve WER by up to 9.3%relative.

Mask The Bias: Improving Domain-Adaptive Generalization of CTC-based ASR with Internal Language Model Estimation