English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While Large Language Models (LLMs) based agents have successfully mimickedhuman behaviors in various scenarios, the realm of complex, multi-charactersocial interactions within extended contexts remains underexplored. Thechallenge is compounded by privacy concerns, making it difficult to capture andutilize intricate real-life interactions. More importantly, the absence ofquantitative evaluation methods hampers the pursuit of high-quality agentinteractions, often leading to interactions that are limited in informativenessand expressiveness, characterized by superficial small talk without clearintentions. In this work, we leverage the rules of Tabletop Role-Playing Games(TRPG) to create an environment conducive to complex, context-richinteractions, emphasizing informativeness and expressiveness. This virtualsetting alleviates privacy concerns and motivates agents to engage inmeaningful, high-quality interactions as part of their in-game objectives. Toassess these interactions, we introduce the Agent interaction Evaluationframework (AntEval), targeting the qualitative evaluation of interactioninformativeness and expressiveness. Specifically, we propose two novelevaluation metrics: Information Exchanging Precision (IEP) and InteractionExpressiveness Gap (IEG). These metrics are designed to assess interactions inscenarios focused on information exchange and intention expression,respectively. Our experimental results demonstrate the effectiveness of thesemetrics in evaluating interaction quality. Notably, we identify significantareas for improvement in LLMs regarding social interactions, as highlighted byour metrics. We believe AntEval will guide further exploration in complex agentinteractions, bringing them closer to emulating real human behavior andenhancing their integration and utility in real-world applications.

AntEval: Quantitatively Evaluating Informativeness and Expressiveness of Agent Social Interactions