English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deployed artificial intelligence (AI) often impacts humans, and there is noone-size-fits-all metric to evaluate these tools. Human-centered evaluation ofAI-based systems combines quantitative and qualitative analysis and humaninput. It has been explored to some depth in the explainable AI (XAI) andhuman-computer interaction (HCI) communities. Gaps remain, but the basicunderstanding that humans interact with AI and accompanying explanations, andthat humans' needs -- complete with their cognitive biases and quirks -- shouldbe held front and center, is accepted by the community. In this paper, we drawparallels between the relatively mature field of XAI and the rapidly evolvingresearch boom around large language models (LLMs). Accepted evaluative metricsfor LLMs are not human-centered. We argue that many of the same paths tread bythe XAI community over the past decade will be retread when discussing LLMs.Specifically, we argue that humans' tendencies -- again, complete with theircognitive biases and quirks -- should rest front and center when evaluatingdeployed LLMs. We outline three developed focus areas of human-centeredevaluation of XAI: mental models, use case utility, and cognitive engagement,and we highlight the importance of exploring each of these concepts for LLMs.Our goal is to jumpstart human-centered LLM evaluation.

Who's Thinking? A Push for Human-Centered Evaluation of LLMs using the XAI Playbook