English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Several benchmarks have been built with heavy investment in resources totrack our progress in NLP. Thousands of papers published in response to thosebenchmarks have competed to top leaderboards, with models often surpassinghuman performance. However, recent studies have shown that models triumph overseveral popular benchmarks just by overfitting on spurious biases, withouttruly learning the desired task. Despite this finding, benchmarking, whiletrying to tackle bias, still relies on workarounds, which do not fully utilizethe resources invested in benchmark creation, due to the discarding of lowquality data, and cover limited sets of bias. A potential solution to theseissues -- a metric quantifying quality -- remains underexplored. Inspired bysuccessful quality indices in several domains such as power, food, and water,we take the first step towards a metric by identifying certain languageproperties that can represent various possible interactions leading to biasesin a benchmark. We look for bias related parameters which can potentially helppave our way towards the metric. We survey existing works and identifyparameters capturing various properties of bias, their origins, types andimpact on performance, generalization, and robustness. Our analysis spans overdatasets and a hierarchy of tasks ranging from NLI to Summarization, ensuringthat our parameters are generic and are not overfitted towards a specific taskor dataset. We also develop certain parameters in this process.

A Survey of Parameters Associated with the Quality of Benchmarks in NLP