English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

In the last decade, the Winograd Schema Challenge (WSC) has become a centralaspect of the research community as a novel litmus test. Consequently, the WSChas spurred research interest because it can be seen as the means to understandhuman behavior. In this regard, the development of new techniques has madepossible the usage of Winograd schemas in various fields, such as the design ofnovel forms of CAPTCHAs.  Work from the literature that established a baseline for human adultperformance on the WSC has shown that not all schemas are the same, meaningthat they could potentially be categorized according to their perceivedhardness for humans. In this regard, this \textit{hardness-metric} could beused in future challenges or in the WSC CAPTCHA service to differentiatebetween Winograd schemas.  Recent work of ours has shown that this could be achieved via the design ofan automated system that is able to output the hardness-indexes of Winogradschemas, albeit with limitations regarding the number of schemas it could beapplied on. This paper adds to previous research by presenting a new systemthat is based on Machine Learning (ML), able to output the hardness of anyWinograd schema faster and more accurately than any other previously usedmethod. Our developed system, which works within two different approaches,namely the random forest and deep learning (LSTM-based), is ready to be used asan extension of any other system that aims to differentiate between Winogradschemas, according to their perceived hardness for humans. At the same time,along with our developed system we extend previous work by presenting theresults of a large-scale experiment that shows how human performance variesacross Winograd schemas.

Experience and Prediction: A Metric of Hardness for a Novel Litmus Test