English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The evaluation of large language models is an essential task in the field oflanguage understanding and generation. As language models continue to advance,the need for effective benchmarks to assess their performance has becomeimperative. In the context of Traditional Chinese, there is a scarcity ofcomprehensive and diverse benchmarks to evaluate the capabilities of languagemodels, despite the existence of certain benchmarks such as DRCD, TTQA, CMDQA,and FGC dataset. To address this gap, we propose a novel set of benchmarks thatleverage existing English datasets and are tailored to evaluate language modelsin Traditional Chinese. These benchmarks encompass a wide range of tasks,including contextual question-answering, summarization, classification, andtable understanding. The proposed benchmarks offer a comprehensive evaluationframework, enabling the assessment of language models' capabilities acrossdifferent tasks. In this paper, we evaluate the performance of GPT-3.5,Taiwan-LLaMa-v1.0, and Model 7-C, our proprietary model, on these benchmarks.The evaluation results highlight that our model, Model 7-C, achievesperformance comparable to GPT-3.5 with respect to a part of the evaluatedcapabilities. In an effort to advance the evaluation of language models inTraditional Chinese and stimulate further research in this field, we haveopen-sourced our benchmark and opened the model for trial.

Advancing the Evaluation of Traditional Chinese Language Models: Towards a Comprehensive Benchmark Suite