English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large Language Models (LLMs) hold immense potential to generate syntheticdata of high quality and utility, which has numerous applications fromdownstream model training to practical data utilisation. However, contemporarymodels, despite their impressive capacities, consistently struggle to produceboth coherent and diverse data. To address the coherency issue, we introducecontrastive expert guidance, where the difference between the logitdistributions of fine-tuned and base language models is emphasised to ensuredomain adherence. In order to ensure diversity, we utilise existing real andsynthetic examples as negative prompts to the model. We deem this dual-prongedapproach to logit reshaping as STEER: Semantic Text Enhancement via EmbeddingRepositioning. STEER operates at inference-time and systematically guides theLLMs to strike a balance between adherence to the data distribution (ensuringsemantic fidelity) and deviation from prior synthetic examples or existing realdatasets (ensuring diversity and authenticity). This delicate balancing act isachieved by dynamically moving towards or away from chosen representations inthe latent space. STEER demonstrates improved performance over previoussynthetic data generation techniques, exhibiting better balance between datadiversity and coherency across three distinct tasks: hypothesis generation,toxic and non-toxic comment generation, and commonsense reasoning taskgeneration. We demonstrate how STEER allows for fine-tuned control over thediversity-coherency trade-off via its hyperparameters, highlighting itsversatility.

Steering Language Generation: Harnessing Contrastive Expert Guidance and Negative Prompting for Coherent and Diverse Synthetic Data Generation