English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The success of large language models has shifted the evaluation paradigms innatural language processing (NLP). The community's interest has drifted towardscomparing NLP models across many tasks, domains, and datasets, often at anextreme scale. This imposes new engineering challenges: efforts in constructingdatasets and models have been fragmented, and their formats and interfaces areincompatible. As a result, it often takes extensive (re)implementation effortsto make fair and controlled comparisons at scale.  Catwalk aims to address these issues. Catwalk provides a unified interface toa broad range of existing NLP datasets and models, ranging from both canonicalsupervised training and fine-tuning, to more modern paradigms like in-contextlearning. Its carefully-designed abstractions allow for easy extensions to manyothers. Catwalk substantially lowers the barriers to conducting controlledexperiments at scale. For example, we finetuned and evaluated over 64 models onover 86 datasets with a single command, without writing any code. Maintained bythe AllenNLP team at the Allen Institute for Artificial Intelligence (AI2),Catwalk is an ongoing open-source effort: https://github.com/allenai/catwalk.

Catwalk: A Unified Language Model Evaluation Framework for Many Datasets