English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Instruction-tuned Large Language Models (LLMs) have exhibited impressivelanguage understanding and the capacity to generate responses that followspecific prompts. However, due to the computational demands associated withtraining these models, their applications often adopt a zero-shot setting. Inthis paper, we evaluate the zero-shot performance of two publicly accessibleLLMs, ChatGPT and OpenAssistant, in the context of six Computational SocialScience classification tasks, while also investigating the effects of variousprompting strategies. Our experiments investigate the impact of promptcomplexity, including the effect of incorporating label definitions into theprompt; use of synonyms for label names; and the influence of integrating pastmemories during foundation model training. The findings indicate that in azero-shot setting, current LLMs are unable to match the performance of smaller,fine-tuned baseline transformer models (such as BERT-large). Additionally, wefind that different prompting strategies can significantly affectclassification accuracy, with variations in accuracy and F1 scores exceeding10\%.

Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Science