English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Large language models (LLMs) have achieved remarkable performance in variousevaluation benchmarks. However, concerns are raised about potential datacontamination in their considerable volume of training corpus. Moreover, thestatic nature and fixed complexity of current benchmarks may inadequately gaugethe advancing capabilities of LLMs. In this paper, we introduce DyVal, ageneral and flexible protocol for dynamic evaluation of LLMs. Based on ourframework, we build graph-informed DyVal by leveraging the structural advantageof directed acyclic graphs to dynamically generate evaluation samples withcontrollable complexities. DyVal generates challenging evaluation sets onreasoning tasks including mathematics, logical reasoning, and algorithmproblems. We evaluate various LLMs ranging from Flan-T5-large to GPT-3.5-Turboand GPT-4. Experiments show that LLMs perform worse in DyVal-generatedevaluation samples with different complexities, highlighting the significanceof dynamic evaluation. We also analyze the failure cases and results ofdifferent prompting methods. Moreover, DyVal-generated samples are not onlyevaluation sets, but also helpful data for fine-tuning to improve theperformance of LLMs on existing benchmarks. We hope that DyVal can shed lighton future evaluation research of LLMs. Code is available at:https://github.com/microsoft/promptbench.

DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks