English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Language models generate texts by successively predicting probabilitydistributions for next tokens given past ones. A growing field of interesttries to leverage external information in the decoding process so that thegenerated texts have desired properties, such as being more natural, non toxic,faithful, or having a specific writing style. A solution is to use a classifierat each generation step, resulting in a cooperative environment where theclassifier guides the decoding of the language model distribution towardsrelevant texts for the task at hand. In this paper, we examine three familiesof (transformer-based) discriminators for this specific task of cooperativedecoding: bidirectional, left-to-right and generative ones. We evaluate thepros and cons of these different types of discriminators for cooperativegeneration, exploring respective accuracy on classification tasks along withtheir impact on the resulting sample quality and computational performances. Wealso provide the code of a batched implementation of the powerful cooperativedecoding strategy used for our experiments, the Monte Carlo Tree Search,working with each discriminator for Natural Language Generation.

Which Discriminator for Cooperative Text Generation?