English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The quest for human imitative AI has been an enduring topic in AI researchsince its inception. The technical evolution and emerging capabilities of thelatest cohort of large language models (LLMs) have reinvigorated the subjectbeyond academia to the cultural zeitgeist. While recent NLP evaluationbenchmark tasks test some aspects of human-imitative behaviour (e.g.,BIG-bench's 'human-like behavior' tasks), few, if not none, examine creativeproblem solving abilities. Creative problem solving in humans is a well-studiedtopic in cognitive neuroscience with standardized tests that predominantly usethe ability to associate (heterogeneous) connections among clue words as ametric for creativity. Exposure to misleading stimuli - distractors dubbed redherrings - impede human performance in such tasks via the fixation effect andEinstellung paradigm. In cognitive neuroscience studies, such fixations areexperimentally induced by pre-exposing participants to orthographically similarincorrect words to subsequent word-fragments or clues. The popular British quizshow Only Connect's Connecting Wall segment essentially mimics Mednick's RemoteAssociates Test (RAT) formulation with built-in, deliberate red herrings, whichmakes it an ideal proxy dataset to explore and study fixation effect andEinstellung paradigm from cognitive neuroscience in LLMs. In this paper wepresent the novel Only Connect Wall (OCW) dataset and report results from ourevaluation of selected pre-trained language models and LLMs on creative problemsolving tasks like grouping clue words by heterogeneous connections, andidentifying correct open knowledge domain connections in respective groups. Wesynthetically generate two additional datasets: OCW-Randomized, OCW-WordNet tofurther analyze our red-herrings hypothesis in language models. The code andlink to the dataset are available at https://github.com/TaatiTeam/OCW.

Large Language Models are Fixated by Red Herrings: Exploring Creative Problem Solving and Einstellung Effect using the Only Connect Wall Dataset