English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Variational Auto-Encoder (VAE) has been widely adopted in text generation.Among many variants, recurrent VAE learns token-wise latent variables with eachconditioned on the preceding ones, which captures sequential variability betterin the era of RNN. However, it is unclear how to incorporate such recurrentdynamics into the recently dominant Transformer due to its parallelism. In thiswork, we propose TRACE, a Transformer-based recurrent VAE structure. TRACEimposes recurrence on segment-wise latent variables with arbitrarily separatedtext segments and constructs the posterior distribution with residualparameterization. Besides, we design an acceleration method by approximatingidempotent matrices, which allows parallelism while maintaining the conditionaldependence of latent variables. We demonstrate that TRACE could enhance theentanglement of each segment and preceding latent variables and deduce anon-zero lower bound of the KL term, providing a theoretical guarantee ofgeneration diversity. Experiments on two unconditional and one conditionalgeneration tasks show that TRACE achieves significantly improved diversitywhile maintaining satisfactory generation quality.

Recurrence Boosts Diversity! Revisiting Recurrent Latent Variable in Transformer-Based Variational AutoEncoder for Diverse Text Generation