English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large language models (LLMs) such as ChatGPT and GPT-4 have shown impressiveperformance in complex reasoning tasks. However, it is difficult to knowwhether the models are reasoning based on deep understandings of truth andlogic, or leveraging their memorized patterns in a relatively superficial way.In this work, we explore testing LLMs' reasoning by engaging with them in adebate-like conversation, where given a question, the LLM and the user need todiscuss to make the correct decision starting from opposing arguments. Uponmitigating the Clever Hans effect, our task requires the LLM to not onlyachieve the correct answer on its own, but also be able to hold and defend itsbelief instead of blindly believing or getting misled by the user's (invalid)arguments and critiques, thus testing in greater depth whether the LLM graspsthe essence of the reasoning required to solve the problem. Across a range ofcomplex reasoning benchmarks spanning math, commonsense, logic and BIG-Benchtasks, we find that despite their impressive performance as reported inexisting work on generating correct step-by-step solutions in the beginning,LLMs like ChatGPT cannot maintain their beliefs in truth for a significantportion of examples when challenged by oftentimes absurdly invalid arguments.Our work points to danger zones of model alignment, and also suggests morecareful treatments and interpretations of the recent findings that LLMs canimprove their responses based on feedback.

Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate