English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Language models have become very popular recently and many claims have beenmade about their abilities, including for commonsense reasoning. Given theincreasingly better results of current language models on previous staticbenchmarks for commonsense reasoning, we explore an alternative dialecticalevaluation. The goal of this kind of evaluation is not to obtain an aggregateperformance value but to find failures and map the boundaries of the system.Dialoguing with the system gives the opportunity to check for consistency andget more reassurance of these boundaries beyond anecdotal evidence. In thispaper we conduct some qualitative investigations of this kind of evaluation forthe particular case of spatial reasoning (which is a fundamental aspect ofcommonsense reasoning). We conclude with some suggestions for future work bothto improve the capabilities of language models and to systematise this kind ofdialectical evaluation.

Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMs