English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Reinforcement Learning from Human Feedback (RLHF) is a vital strategy forenhancing model safety in language models. However, annotating preference datafor RLHF is a resource-intensive and creativity-demanding process, whileautomatic generation methods face limitations in data diversity and quality. Inresponse, we present Safer-Instruct, a novel pipeline for semi-automaticallyconstructing large-scale preference datasets. Our approach leverages reversedinstruction tuning, instruction induction, and expert model evaluation toefficiently generate high-quality preference data without human annotators. Weevaluate Safer-Instruct using LLaMA for instruction induction and GPT-4 as anexpert model, generating approximately 10K preference samples. Finetuning anAlpaca model on this dataset demonstrates improved harmlessness whilemaintaining competitive performance on conversation and downstream tasks.Safer-Instruct addresses the challenges in preference data acquisition,advancing the development of safer and more responsible AI systems. Our codeand data are available at https://github.com/uscnlp-lime/safer-instruct

Safer-Instruct: Aligning Language Models with Automated Preference Data