Open AccessMy Computer Is an Honor Student — But How Intelligent Is It? Standardized Tests as a Measure of AI
Author(s) -
Clark Peter,
Etzioni Oren
Publication year - 2016
Publication title -
ai magazine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.597
H-Index - 79
eISSN - 2371-9621
pISSN - 0738-4602
DOI - 10.1609/aimag.v37i1.2636
Subject(s) - measure (data warehouse) , computer science , task (project management) , turing test , standardized test , artificial intelligence , test (biology) , key (lock) , focus (optics) , simple (philosophy) , turing , data science , machine learning , mathematics education , programming language , data mining , psychology , engineering , systems engineering , computer security , optics , epistemology , biology , paleontology , philosophy , physics
Given the well‐known limitations of the Turing test, there is a need for objective tests to both focus attention on, and measure progress toward, the goals of AI. In this paper we argue that machine performance on standardized tests should be a key component of any new measure of AI, because attaining a high level of performance requires solving significant AI problems involving language understanding and world modeling — critical skills for any machine that lays claim to intelligence. In addition, standardized tests have all the basic requirements of a practical test: they are accessible, easily comprehensible, clearly measurable, and offer a graduated progression from simple tasks to those requiring deep understanding of the world. Here we propose this task as a challenge problem for the community, summarize our state‐of‐the‐art results on math and science tests, and provide supporting data sets ( www.allenai.org ).