Increasing Learner Engagement in Online Learning through Use of Interactive Feedback: Results of a Pilot Study

Technologies for eLearning continue to evolve and provide additional mechanisms for teaching and facilitating learner engagement. As the number of engineering courses and programs provided in an online format continues to increase, the need for evaluating the efficacy of these eLearning tools also increases. One of the main concerns in online learning is learner persistence, so technologies and pedagogies that support persistence are especially important. A growing body of literature suggests that when students feel connected and supported, they are more likely to continue with a program. Finding ways to accomplish this support and engagement for online programs then is a significant factor in program delivery. In this paper we report on the results of a pilot study that examined the use of text-based and interactive feedback using the framework of the Community of Inquiry (COI) Model. This model includes elements of cognitive presence, social presence and teaching presence which are associated with student engagement, connectedness and support. The pilot was conducted over two semesters in an online engineering course at the University of Cincinnati. Students were provided both traditional text-based feedback on assignments and interactive feedback. Surveys were administered to measure students’ response to both forms of feedback and to gauge how both forms of feedback impacted elements of the COI Model. The pilot study indicates that feedback has the greatest impact on teacher presence with smaller association with social presence and cognitive presence. The use of interactive feedback was certainly appreciated by the students but perhaps of more importance is timeliness of feedback and personalizing feedback to the individual. Student engagement is improved with the use of interactive feedback but the effort to provide this type of feedback may not be warranted for all courses. Introduction Garrison, Anderson, and Archer have developed a theoretical framework for representing the process of learning and creating meaning. The community of inquiry model includes three interdependent elements – cognitive presence, social presence, and teaching presence. Cognitive presence refers to the extent that students are able to construct meaning and knowledge through sustained communication. Social presence is described as the propensity for students in a learning setting to authentically present their personal traits and attributes into that setting. Teaching presence is described as the design, implementation and oversight of instruction and instructional processes in order to achieve prescribed learning outcomes. A significant aspect of each of these three elements relies on interactions between the instructor and the students so pedagogies or technologies that significantly modify the nature and extent of interactions can have effects on the model. Table 1 illustrates methods for coding student behaviors that are indicative of the three elements of the model. Table 1 Community of Inquiry Coding Template Elements Categories Indicators (example) Cognitive Presence Q4, Q5 Triggering Event Sense of puzzlement Exploration Information exchange Integration Connecting ideas Resolution Apply new ideas Social Presence Q3, Emotional Expression Emoticons Open Communication Risk-free expression Group Cohesion Encouraging collaboration Teaching Presence Q1, Q2, Q6, Q7, Q8, Q9 Instructional Management Defining & initiating discussion topics Building Understanding Sharing personal meaning Direct Instruction Focusing Discussion (After Garrison, Anderson, & Archer, 2000, p. 4) Table 1 also lists the survey questions (discussed in the Results section) that are associated with specific elements of the COI model. As technology changes, so does our need to explore and evaluate effectiveness of these technologies. This is particularly true of online courses and programs that rely on a variety of technologies for their effective development and implementation. Feedback from instructor to students affects all elements of the COI model and is dependent on technology in online courses. In previous studies, the primary point of discussion has focused on the technology employed to deliver feedback as Chen, Whittinghill and Kadlow discuss in their review of clickers for rapid feedback. While exploring the adoption and acceptance of such technologies is useful, so is the importance of studying the impact of feedback, and its form, on student learning using the COI framework. Previous studies (Ice, Curtis, Phillips & Wells; Dias & Trumpy) offer a deeper understanding on students’ experience and sense of social presence as a result of audio feedback strategies, reflecting students’ value for effectiveness and efficiency in the context of audiobased feedback. Feedback to Improve Students’ Perception of Engagement Harper proposed that instructors are in search of meaningful methods for promoting interactivity and engagement. Proper attention to instructional design informs us that appropriate and regular feedback is a necessity for meaningful learning. When considering feedback we should consider both the mechanisms we use to provide feedback to students, particularly in online courses, as well as students’ perceptions of quality feedback, which is likely to vary from one student to the next. Using audio as a means to provide feedback has been employed as early as the 1960s and research regarding the effectiveness of audio feedback as an instructional tool has also been investigated for many years (Tanner and McGrew). In recent studies comparing the use of text versus audio-based feedback (Haper; Halupa & Bolliger) findings indicated that students valued both methods of feedback (text and audio), but primarily valued audio-based feedback for the added ability to express nuance through voice and intonation. Other critical studies include a study by Halupa & Bolliger that supported students’ valuation of the feedback, and provided further data to illuminate students’ perceptions and preferences that prevented them from engaging with the formative feedback provided by instructors. Students’ perceptions do seem to correlate to their participation and interest around engaging with feedback in their courses. This may be due to the nature of written feedback as being perceived as a one-way (Stone) experience from instructor-to-student, even though the act of reading is required to complete the feedback loop. As with other research studies, Stone confirms the ongoing rhetoric surrounding feedback methods in online courses in that it is necessary and more than often, welcomed by students who are generally open to the experience. Student Perception of Feedback in Online Learning Moore’s (1993) theory of transactional distance proposed a structure for examining communications in online environments. Within this framework, Moore (1993) proposes quality over quantity for a productive and successful interaction online. While this theoretical framework can provide a guide for ways in which we can consider the impact of feedback (i.e. frequency, tone, method), we should also consider how students perceive these interactions. In a more recent study (Dias & Trumpy) explored the use of formative feedback via written and audio feedback in an online classroom. From this study, Dias & Trumpy found that students generally found audio-based feedback to be more accessible. These studies shed light on perceived benefits of efficiency but have not used the lens of the Community of Inquiry and fail to measure whether students feel more ‘connected’ when a particular method of feedback is used. Lamport & Bartolo, in a recent study focused on exploring student perceptions of online instructional practices, discovered elements that students perceived as having promoted a stronger sense of community, which included timely teacher feedback. It is evident that feedback is necessary, but our understanding of how students perceive it in the context of ‘presence’ related to the Community of Inquiry, may help us to better understand what feedback should look, feel, sound and be like to make its greatest impact. Methods for Investigating Use of Interactive (Audio) Feedback Two distinct but related investigations were performed to assess the impact interactive feedback had on learner engagement using the COI model as a framework. Both investigations were “housed” within a course taken by upperclassmen and first year graduate students. This particular course, Effectiveness in Technical Organizations, is an elective course that addresses professional skills such as communication, team work and leadership. Assessment of student learning is measured in large part by reflective papers written by the students. The first investigation was completed during the 2013-2014 academic year and included 40 students. During the first half of the semester students received text-based feedback on their written assignments (this method had been used by the instructor for the previous five years). During the second half of the semester, students received interactive feedback on their written assignments. In all cases the feedback was provided to the students via the university’s leaning management system. Interactive feedback was created by first making written comments on a student’s assignment then using Screencast-o-matic to record audio and the instructor’s computer screen to expound on the written comments and make summary comments. Figure 1 provides a screenshot of what a student would see to “play” the interactive feedback. Students were given a survey at the mid-term (when they had received text-based feedback) and conclusion (when they had received interactive feedback) of the course. The survey was adapted by permission from the COI database at Athabasca University. This survey is reported in Appendix A. In addition students were given a traditional end-of-course survey that also included open-ended