English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A final exam in machine learning at a top institution such as MIT, Harvard,or Cornell typically takes faculty days to write, and students hours to solve.We demonstrate that large language models pass machine learning finals at ahuman level, on finals available online after the models were trained, andautomatically generate new human-quality final exam questions in seconds.Previous work has developed program synthesis and few-shot learning methods tosolve university-level problem set questions in mathematics and STEM courses.In this work, we develop and compare methods that solve final exams, whichdiffer from problem sets in several ways: the questions are longer, havemultiple parts, are more complicated, and span a broader set of topics. Wecurate a dataset and benchmark of questions from machine learning final examsavailable online and code for answering these questions and generating newquestions. We show how to generate new questions from other questions andcourse notes. For reproducibility and future research on this final exambenchmark, we use automatic checkers for multiple-choice, numeric, andquestions with expression answers. We perform ablation studies comparingzero-shot learning with few-shot learning and chain-of-thought prompting usingGPT-3, OPT, Codex, and ChatGPT across machine learning topics and find thatfew-shot learning methods perform best. We highlight the transformativepotential of language models to streamline the writing and solution oflarge-scale assessments, significantly reducing the workload from human days tomere machine seconds. Our results suggest that rather than banning largelanguage models such as ChatGPT in class, instructors should teach students toharness them by asking students meta-questions about correctness, completeness,and originality of the responses generated, encouraging critical thinking inacademic studies.

From Human Days to Machine Seconds: Automatically Answering and Generating Machine Learning Final Exams