English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The misuse of large language models (LLMs) has garnered significant attentionfrom the general public and LLM vendors. In response, efforts have been made toalign LLMs with human values and intent use. However, a particular type ofadversarial prompts, known as jailbreak prompt, has emerged and continuouslyevolved to bypass the safeguards and elicit harmful content from LLMs. In thispaper, we conduct the first measurement study on jailbreak prompts in the wild,with 6,387 prompts collected from four platforms over six months. Leveragingnatural language processing technologies and graph-based community detectionmethods, we discover unique characteristics of jailbreak prompts and theirmajor attack strategies, such as prompt injection and privilege escalation. Wealso observe that jailbreak prompts increasingly shift from public platforms toprivate ones, posing new challenges for LLM vendors in proactive detection. Toassess the potential harm caused by jailbreak prompts, we create a question setcomprising 46,800 samples across 13 forbidden scenarios. Our experiments showthat current LLMs and safeguards cannot adequately defend jailbreak prompts inall scenarios. Particularly, we identify two highly effective jailbreak promptswhich achieve 0.99 attack success rates on ChatGPT (GPT-3.5) and GPT-4, andthey have persisted online for over 100 days. Our work sheds light on thesevere and evolving threat landscape of jailbreak prompts. We hope our studycan facilitate the research community and LLM vendors in promoting safer andregulated LLMs.

"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models