English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Prompting serves as the major way humans interact with Large Language Models(LLM). Commercial AI systems commonly define the role of the LLM in systemprompts. For example, ChatGPT uses ``You are a helpful assistant'' as part ofits default system prompt. Despite current practices of adding personas tosystem prompts, it remains unclear how different personas affect a model'sperformance on objective tasks. In this study, we present a systematicevaluation of personas in system prompts. We curate a list of 162 rolescovering 6 types of interpersonal relationships and 8 domains of expertise.Through extensive analysis of 4 popular families of LLMs and 2,410 factualquestions, we demonstrate that adding personas in system prompts does notimprove model performance across a range of questions compared to the controlsetting where no persona is added. Nevertheless, further analysis suggests thatthe gender, type, and domain of the persona can all influence the resultingprediction accuracies. We further experimented with a list of persona searchstrategies and found that, while aggregating results from the best persona foreach question significantly improves prediction accuracy, automaticallyidentifying the best persona is challenging, with predictions often performingno better than random selection. Overall, our findings suggest that whileadding a persona may lead to performance gains in certain settings, the effectof each persona can be largely random. Code and data are available athttps://github.com/Jiaxin-Pei/Prompting-with-Social-Roles.

When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models