English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper investigates the rational thinking capability of Large LanguageModels (LLMs) in multi-round argumentative debates by exploring the impact offallacious arguments on their logical reasoning performance. More specifically,we present Logic Competence Measurement Benchmark (LOGICOM), a diagnosticbenchmark to assess the robustness of LLMs against logical fallacies. LOGICOMinvolves two agents: a persuader and a debater engaging in a multi-round debateon a controversial topic, where the persuader tries to convince the debater ofthe correctness of its claim. First, LOGICOM assesses the potential of LLMs tochange their opinions through reasoning. Then, it evaluates the debater'sperformance in logical reasoning by contrasting the scenario where thepersuader employs logical fallacies against one where logical reasoning isused. We use this benchmark to evaluate the performance of GPT-3.5 and GPT-4using a dataset containing controversial topics, claims, and reasons supportingthem. Our findings indicate that both GPT-3.5 and GPT-4 can adjust theiropinion through reasoning. However, when presented with logical fallacies,GPT-3.5 and GPT-4 are erroneously convinced 41% and 69% more often,respectively, compared to when logical reasoning is used. Finally, we introducea new dataset containing over 5k pairs of logical vs. fallacious arguments. Thesource code and dataset of this work are made publicly available.

How susceptible are LLMs to Logical Fallacies?