English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We analyzed effectiveness of three generative pre-trained transformer (GPT)models in answering multiple-choice question (MCQ) assessments, often involvingshort snippets of code, from introductory and intermediate programming coursesat the postsecondary level. This emerging technology stirs countlessdiscussions of its potential uses (e.g., exercise generation, code explanation)as well as misuses in programming education (e.g., cheating). However, thecapabilities of GPT models and their limitations to reason about and/or analyzecode in educational settings have been under-explored. We evaluated severalOpenAI's GPT models on formative and summative MCQ assessments from threePython courses (530 questions). We found that MCQs containing code snippets arenot answered as successfully as those that only contain natural language. Whilequestions requiring to fill-in a blank in the code or completing a naturallanguage statement about the snippet are handled rather successfully, MCQs thatrequire analysis and/or reasoning about the code (e.g., what is true/falseabout the snippet, or what is its output) appear to be the most challenging.These findings can be leveraged by educators to adapt their instructionalpractices and assessments in programming courses, so that GPT becomes avaluable assistant for a learner as opposed to a source of confusion and/orpotential hindrance in the learning process.

Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code