English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large language models have recently made tremendous progress in a variety ofaspects, e.g., cross-task generalization, instruction following.Comprehensively evaluating the capability of large language models in multipletasks is of great importance. In this paper, we propose M3KE, a MassiveMulti-Level Multi-Subject Knowledge Evaluation benchmark, which is developed tomeasure knowledge acquired by Chinese large language models by testing theirmultitask accuracy in zero- and few-shot settings. We have collected 20,477questions from 71 tasks. Our selection covers all major levels of Chineseeducation system, ranging from the primary school to college, as well as a widevariety of subjects, including humanities, history, politics, law, education,psychology, science, technology, art and religion. All questions aremultiple-choice questions with four options, hence guaranteeing a standardizedand unified assessment process. We've assessed a number of state-of-the-artopen-source Chinese large language models on the proposed benchmark. The sizeof these models varies from 335M to 130B parameters. Experiment resultsdemonstrate that they perform significantly worse than GPT-3.5 that reaches anaccuracy of ~ 48% on M3KE. The dataset is available athttps://github.com/tjunlp-lab/M3KE.

M3KE: A Massive Multi-Level Multi-Subject Knowledge Evaluation Benchmark for Chinese Large Language Models