English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Large language models (LLMs) are effective at answering questions that areclearly asked. However, when faced with ambiguous queries they can actunpredictably and produce incorrect outputs. This underscores the need for thedevelopment of intelligent agents capable of asking clarification questions toresolve ambiguities effectively. This capability requires complexunderstanding, state tracking, reasoning and planning over multipleconversational turns. However, directly measuring this can be challenging. Inthis paper, we offer a surrogate problem which assesses an LLMs's capability todeduce an entity unknown to itself, but revealed to a judge, by asking thejudge a series of queries. This entity-deducing game can serve as an evaluationframework to probe the conversational reasoning and planning capabilities oflanguage models. We systematically evaluate various LLMs and discoversignificant differences in their performance on this task. We find that strongLLMs like GPT-4 outperform human players by a large margin. We further employBehavior Cloning (BC) to examine whether a weaker model is capable of imitatinga stronger model and generalizing to data or domains, using only thedemonstrations from a stronger model. We finally propose to use ReinforcementLearning to enhance reasoning and planning capacity of Vicuna models throughepisodes of game playing, which lead to significant performance improvement. Wehope that this problem offers insights into how autonomous agents could betrained to behave more intelligently in ambiguous circumstances.

The Entity-Deduction Arena: A playground for probing the conversational reasoning and planning capabilities of LLMs