English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Since the natural language processing (NLP) community started to make largelanguage models (LLMs), such as GPT-4, act as a critic to evaluate the qualityof generated texts, most of them only train a critique generation model of aspecific scale on specific datasets. We argue that a comprehensiveinvestigation on the key factor of LLM-based evaluation models, such as scalingproperties, is lacking, so that it is still inconclusive whether these modelshave potential to replace GPT-4's evaluation in practical scenarios. In thispaper, we propose a new critique generation model called CritiqueLLM, whichincludes a dialogue-based prompting method for high-quality referenced /reference-free evaluation data. Experimental results show that our model canachieve comparable evaluation performance to GPT-4 especially in system-levelcorrelations, and even outperform GPT-4 in 3 out of 8 tasks in a challengingreference-free setting. We conduct detailed analysis to show promising scalingproperties of our model in the quality of generated critiques. We alsodemonstrate that our generated critiques can act as scalable feedback todirectly improve the generation quality of LLMs.

CritiqueLLM: Scaling LLM-as-Critic for Effective and Explainable Evaluation of Large Language Model Generation