English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Users seek security & privacy (S&P) advice from online resources, includingtrusted websites and content-sharing platforms. These resources help usersunderstand S&P technologies and tools and suggest actionable strategies. LargeLanguage Models (LLMs) have recently emerged as trusted information sources.However, their accuracy and correctness have been called into question. Priorresearch has outlined the shortcomings of LLMs in answering multiple-choicequestions and user ability to inadvertently circumvent model restrictions(e.g., to produce toxic content). Yet, the ability of LLMs to provide reliableS&P advice is not well-explored. In this paper, we measure their ability torefute popular S&P misconceptions that the general public holds. We first studyrecent academic literature to curate a dataset of over a hundred S&P-relatedmisconceptions across six different topics. We then query two popular LLMs(Bard and ChatGPT) and develop a labeling guide to evaluate their responses tothese misconceptions. To comprehensively evaluate their responses, we furtherapply three strategies: query each misconception multiple times, generate andquery their paraphrases, and solicit source URLs of the responses. Both modelsdemonstrate, on average, a 21.3% non-negligible error rate, incorrectlysupporting popular S&P misconceptions. The error rate increases to 32.6% whenwe repeatedly query LLMs with the same or paraphrased misconceptions. We alsoexpose that models may partially support a misconception or remainnoncommittal, refusing a firm stance on misconceptions. Our exploration ofinformation sources for responses revealed that LLMs are susceptible toproviding invalid URLs (21.2% for Bard and 67.7% for ChatGPT) or point tounrelated sources (44.2% returned by Bard and 18.3% by ChatGPT).

Can Large Language Models Provide Security & Privacy Advice? Measuring the Ability of LLMs to Refute Misconceptions