Board 69: Do Adaptive Lessons for Pre-class Experience Improve Flipped Learning?

In a meta-study of STEM courses, use of active learning over traditional lecturing indicated an increase of 0.47 standard deviations on exams and concept inventories. One pedagogy that uses active learning is the flipped classroom, where the initial exposure to the content is obtained outside the classroom via videos, text, online discussion, and assessment. During class time, active learning techniques such as peer-to-peer instruction and solving of applied problems are used. In a prior NSF grant involving a combined dataset from three engineering schools, the authors found that the differences in the cognitive and affective outcomes for blended and flipped formats in a numerical methods course were not statistically significant. The effect sizes measured via Cohen’s d were also negligible to small for these two outcomes. One of the known challenges with the flipped format is the pre-class expectations for the students. Currently for pre-class learning in most flipped courses, instructors assign lecture videos or reading assignments. To ensure that such assignments are done, they are either followed by an online quiz or an in-class quiz at the start of class. However, this approach is the same for all students and does not address the differential needs of students. To improve the quality of the pre-class activities for his numerical methods flipped classroom, the second author developed adaptive lessons using the Smart Sparrow platform under a current NSF grant. By doing so, students had a personalized path for preparation that involved multiple representations such as lecture videos, text, questions, and simulations. The students’ learning was assessed in real time, and depending on their responses, they were taken on alternate paths in the lesson. An analysis of the various metrics available to the instructor from the Smart Sparrow platform demonstrated that the students were actively using the platform. We implemented these adaptive pre-class lessons in the fall 2017 and spring 2018 semesters and discuss preliminary results from the fall 2017 semester in this paper. The results compare three methods – 1) blended instruction 2) flipped instruction without adaptive lessons and 3) flipped instruction with adaptive lessons. The comparisons are based on direct assessment of learning (i.e., final examination), as well as indirect assessments (i.e., student surveys and focus groups). Introduction and Relevant Literature Adaptive learning courseware provides a means of individualized, personalized learning and feedback for students. A “one-size-fits-all” approach is not optimal given individual preferences, interests, needs, and aptitudes, and “Advance Personalized Learning” has been identified as one of the 14 Grand Challenges for Engineering in the 21st Century (National Academy of Engineering). Gartner, a leading IT consulting firm, ranked adaptive learning first on its list of strategic technologies impacting education in 2015, although they stated “A lot of real-world testing remains” (Schaffhauser, 2015). Using computer algorithms, adaptive online courseware analyzes performance data, which is collected as the student utilizes the online learning environment. Based on this, the adaptive courseware determines recommended content or learning activities for the student, provides personalized feedback, and displays real-time progress via dashboards for both the student and instructor. With our prior flipped classroom research with numerical methods coursework, we identified the lack of a personalized approach with the pre-class aspect, where students are expected to learn foundational content before class (via videos or textbooks) prior to the application during class. This served as a motivation for our use of adaptive lessons. In a recent Gates Foundation program with higher education institutions – the Adaptive Learning Market Acceleration program (ALMAP) – modest positive learning results were found with the adaptive implementations in general (Yarnall et al., 2016). In student surveys, 51% of bachelorsdegree students reported positive learning gains with adaptive courseware, although only 33% reported satisfaction with the experience. This recent grant program strongly called for future research in the area of blended classroom implementations of adaptive courseware. Another recent article also called for more research on adaptive learning, after uncovering no significant differences in exam scores between adaptive learning and traditional sections of a course (Murray & Perez, 2015). This makes our current study a needed contribution to the literature. However, there are also recent studies on adaptive tutorials for engineering mechanics courses that point to student satisfaction as well as enhanced outcomes (Prusty & Russell, 2011; Prusty et al., 2011). One study uncovered a reduction in failure rates, an increase in student satisfaction, and highly positive student comments related to the use of the tutorials (Prusty & Russell, 2011). In another study involving an adaptive tutorial on free-body-diagrams, the total number of student comments that identified the tutorial as effective (versus not) was approximately 2:1, with the top reasons for effectiveness given as engaging, immediate feedback, and understanding of concepts. In contrast, the top reasons for ineffectiveness were stated as prefer other methods, confusing or hard to understand, and not enough feedback (Prusty et al., 2011). Thus, results from the literature on adaptive learning outcomes are both minimal and mixed. Given this, our research questions are as follows: 1) Are there achievement differences in a numerical methods course when different methods of instruction are used – a) blended, b) flipped, and c) flipped with adaptive lessons? Are differences evident for underrepresented minorities, females, community college transfers, and Pell Grant recipients? 2) Do students’ perceptions of the classroom environment differ when using these different instructional methods for numerical methods? What are students’ perceptions of flippedclassroom adaptive learning, and are there differences by demographic groups?