English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Previous work adopts large language models (LLMs) as evaluators to evaluatenatural language process (NLP) tasks. However, certain shortcomings, e.g.,fairness, scope, and accuracy, persist for current LLM evaluators. To analyzewhether LLMs can serve as reliable alternatives to humans, we examine thefine-grained alignment between LLM evaluators and human annotators,particularly in understanding the target evaluation tasks and conductingevaluations that meet diverse criteria. This paper explores both conventionaltasks (e.g., story generation) and alignment tasks (e.g., math reasoning), eachwith different evaluation criteria. Our analysis shows that 1) LLM evaluatorscan generate unnecessary criteria or omit crucial criteria, resulting in aslight deviation from the experts. 2) LLM evaluators excel in general criteria,such as fluency, but face challenges with complex criteria, such as numericalreasoning. We also find that LLM-pre-drafting before human evaluation can helpreduce the impact of human subjectivity and minimize annotation outliers inpure human evaluation, leading to more objective evaluation.

Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks