English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

AI developers often apply safety alignment procedures to prevent the misuseof their AI systems. For example, before Meta released Llama 2-Chat, acollection of instruction fine-tuned large language models, they investedheavily in safety training, incorporating extensive red-teaming andreinforcement learning from human feedback. However, it remains unclear howwell safety training guards against model misuse when attackers have access tomodel weights. We explore the robustness of safety training in language modelsby subversively fine-tuning the public weights of Llama 2-Chat. We employlow-rank adaptation (LoRA) as an efficient fine-tuning method. With a budget ofless than $200 per model and using only one GPU, we successfully undo thesafety training of Llama 2-Chat models of sizes 7B, 13B, and 70B. Specifically,our fine-tuning technique significantly reduces the rate at which the modelrefuses to follow harmful instructions. We achieve a refusal rate below 1% forour 70B Llama 2-Chat model on two refusal benchmarks. Our fine-tuning methodretains general performance, which we validate by comparing our fine-tunedmodels against Llama 2-Chat across two benchmarks. Additionally, we present aselection of harmful outputs produced by our models. While there isconsiderable uncertainty about the scope of risks from current models, it islikely that future models will have significantly more dangerous capabilities,including the ability to hack into critical infrastructure, create dangerousbio-weapons, or autonomously replicate and adapt to new environments. We showthat subversive fine-tuning is practical and effective, and hence argue thatevaluating risks from fine-tuning should be a core part of risk assessments forreleasing model weights.

LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B