English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The past several years have witnessed Variational Auto-Encoder's superiorityin various text generation tasks. However, due to the sequential nature of thetext, auto-regressive decoders tend to ignore latent variables and then reduceto simple language models, known as the KL vanishing problem, which wouldfurther deteriorate when VAE is combined with Transformer-based structures. Toameliorate this problem, we propose DELLA, a novel variational Transformerframework. DELLA learns a series of layer-wise latent variables with eachinferred from those of lower layers and tightly coupled with the hidden statesby low-rank tensor product. In this way, DELLA forces these posterior latentvariables to be fused deeply with the whole computation path and henceincorporate more information. We theoretically demonstrate that our method canbe regarded as entangling latent variables to avoid posterior informationdecrease through layers, enabling DELLA to get higher non-zero KL values evenwithout any annealing or thresholding tricks. Experiments on four unconditionaland three conditional generation tasks show that DELLA could better alleviateKL vanishing and improve both quality and diversity compared to several strongbaselines.

Fuse It More Deeply! A Variational Transformer with Layer-Wise Latent Variable Inference for Text Generation