English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In this position paper, I argue that standardized tests for elementaryscience such as SAT or Regents tests are not very good benchmarks for measuringthe progress of artificial intelligence systems in understanding basic science.The primary problem is that these tests are designed to test aspects ofknowledge and ability that are challenging for people; the aspects that arechallenging for AI systems are very different. In particular, standardizedtests do not test knowledge that is obvious for people; none of this knowledgecan be assumed in AI systems. Individual standardized tests also have specificfeatures that are not necessarily appropriate for an AI benchmark. I analyzethe Physics subject SAT in some detail and the New York State Regents Sciencetest more briefly. I also argue that the apparent advantages offered by usingstandardized tests are mostly either minor or illusory. The one major realadvantage is that the significance is easily explained to the public; but Iargue that even this is a somewhat mixed blessing. I conclude by arguing that,first, more appropriate collections of exam style problems could be assembled,and second, that there are better kinds of benchmarks than exam-style problems.In an appendix I present a collection of sample exam-style problems that testkinds of knowledge missing from the standardized tests.

The Limitations of Standardized Science Tests as Benchmarks for Artificial Intelligence Research: Position Paper