English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Multilingual Neural Machine Translation has been showing great success usingtransformer models. Deploying these models is challenging because they usuallyrequire large vocabulary (vocab) sizes for various languages. This limits thespeed of predicting the output tokens in the last vocab projection layer. Toalleviate these challenges, this paper proposes a fast vocabulary projectionmethod via clustering which can be used for multilingual transformers on GPUs.First, we offline split the vocab search space into disjoint clusters given thehidden context vector of the decoder output, which results in much smallervocab columns for vocab projection. Second, at inference time, the proposedmethod predicts the clusters and candidate active tokens for hidden contextvectors at the vocab projection. This paper also includes analysis of differentways of building these clusters in multilingual settings. Our results showend-to-end speed gains in float16 GPU inference up to 25% while maintaining theBLEU score and slightly increasing memory cost. The proposed method speeds upthe vocab projection step itself by up to 2.6x. We also conduct an extensivehuman evaluation to verify the proposed method preserves the quality of thetranslations from the original model.

Fast Vocabulary Projection Method via Clustering for Multilingual Machine Translation on GPU