Work-in-Progress: Automation in Undergraduate Classes: Using Technology to Improve Grading Efficiency, Reliability, and Transparency in Large Classes

Large undergraduate classes offer many challenges relating to scale. This paper describes a suite of automated computer tools developed to assist with these challenges, specifically those relating to grading and performance analysis for either individual students or classes as a whole. While the computer tools developed are independent of any Learning Management System (LMS), they could be adapted to operate more closely with an LMS in other academic environments. The suite of tools in question allow for automated digital rubric generation, collection from students, return to students, and most notably, analysis. Features include the ability to condense several files submitted by one student into a single PDF for review, the ability to execute submitted code in three programming languages (Python 3, MATLAB, and ANSI C) while capturing the output into a PDF, and the ability to track error conditions such as late submission and incorrect file names and automatically assign penalties. Statistical reports are generated for each assignment automatically, providing a window into students’ performance and possible areas of concern. Automated warnings alert the teaching team to potential errors in grading, equity issues (such as one section of the class performing substantially better or worse than another) or opportunities for improvement in the academic process (such as rethinking the pedagogy relating to specific ideas or areas that prove broadly troublesome). These reports streamline instructor workflow and allow for deeper insights into student performance than time would normally allow. The suite of tools was implemented using Visual Basic for Applications (VBA), Python 3, and MySQL databases. The implementation of these automated tools was inexpensive and provided many benefits to the instructors and graders in terms of convenience, time saved, grader accountability, process reliability, and enabling new diagnostic capabilities. Furthermore, cost savings were realized from reduced grader time and from almost eliminating the use of paper to offset the cost of developing the tools. This paper presents details on the tools developed as a part of this effort, preliminary results of the adoption of the tools in a large first-year class, the potential uses of similar tools in other venues, and avenues for future work and development.