English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This article presents a comparative analysis of the ability of two largelanguage model (LLM)-based chatbots, ChatGPT and Bing Chat, recently rebrandedto Microsoft Copilot, to detect veracity of political information. We use AIauditing methodology to investigate how chatbots evaluate true, false, andborderline statements on five topics: COVID-19, Russian aggression againstUkraine, the Holocaust, climate change, and LGBTQ+ related debates. We comparehow the chatbots perform in high- and low-resource languages by using promptsin English, Russian, and Ukrainian. Furthermore, we explore the ability ofchatbots to evaluate statements according to political communication conceptsof disinformation, misinformation, and conspiracy theory, usingdefinition-oriented prompts. We also systematically test how such evaluationsare influenced by source bias which we model by attributing specific claims tovarious political and social actors. The results show high performance ofChatGPT for the baseline veracity evaluation task, with 72 percent of the casesevaluated correctly on average across languages without pre-training. Bing Chatperformed worse with a 67 percent accuracy. We observe significant disparitiesin how chatbots evaluate prompts in high- and low-resource languages and howthey adapt their evaluations to political communication concepts with ChatGPTproviding more nuanced outputs than Bing Chat. Finally, we find that for someveracity detection-related tasks, the performance of chatbots varied dependingon the topic of the statement or the source to which it is attributed. Thesefindings highlight the potential of LLM-based chatbots in tackling differentforms of false information in online environments, but also points to thesubstantial variation in terms of how such potential is realized due tospecific factors, such as language of the prompt or the topic.

In Generative AI we Trust: Can Chatbots Effectively Verify Political Information?