Lessons Learned Using Slack in Engineering Education: An Innovation-based Learning Approach

In Fall 2019, we taught a Cardiovascular Engineering course using a blended approach: a mix between online instruction and face-to-face environment. This course is an interdisciplinary Innovation Based Learning (IBL) class that combines both undergraduate, graduate students, face-to-face and distance education students from different institutions. To foster student collaboration, we decided to use Slack for both instructor-to-student and student-to-student communication. This paper explores the impact of Slack on the course. First, we compiled Slack analytics over the semester, such as the number of messages sent, channel activity, and student engagement. Second, we shared a comprehensive survey about Slack at the end of the course. Finally, we interpreted the latter in light of the former to draw our conclusions and formulate a number of recommendations regarding the use of Slack. Our analytics showed students engaging on Slack throughout the semester, with activity intensifying as course deadlines approached. Other findings indicated students favored direct messages over public ones and a custom engagement metric highlighted the importance of informal Slack channels. Our survey showed that students found Slack had a positive impact on the course. Students appreciated Slack being an all-in-one communication tool, they liked some of its features, and they thought that it struck a good balance between formality and informality. However, students disliked certain aspects of Slack channels, reporting they had trouble finding older messages, and faced various technical issues. Most notably, students reported preferring email over Slack when it came to course announcements. In light of our findings, we formulated a number of recommendations regarding the use of Slack in an educational setting. These include teaching students how to use Slack early on in the semester, having a predefined student-focused channel structure, and practicing “tough love” — that is, deliberately delaying feedback to encourage students to collaborate on solving problems. Introduction In Fall 2019, we taught a class called Cardiovascular Engineering under the Electrical and Computer Engineering Department of North Dakota State University. The class leverages Innovation Based Learning (IBL) [1], a pedagogy similar to Project Based Learning [2] but emphasizing the creation of novel ideas and the development of projects with social impact. Besides having to meet IBL’s requirements, we faced a challenge: our 36 students were based in different locations. Most were spread across two different campuses. Some were taking the course online from various locations across the country. Good communication, inside and outside the class, had to be achieved under these constraints since good communication tools in education are a major requirement to create learning environments that allow students to express creativity inside and outside the class [3]. With that goal in mind, we chose to use Slack (the free version) for class communication, including instructor-to-student and student-to-student communication. In addition, we used a combination of technologies allowing students to explore and embrace various tools used in industry — giving them an edge during recruiting. The technologies ecosystem was hosted on a custom PHP-based web tool called MOOCIBL [4]. It was guided using FOAs geared toward improving student engagement [5]. MOOCIBL and FOAs provided a dynamic environment that enabled the use of IBL tools such as Slack, Prezi, Mentimeter, Tinkercad, or draw.io. Students were monitored on MOOCIBL with a view to the future development of machine learning algorithms supporting IBL [6]. Students engaged in active learning both in and outside class via Massive Open Online Courses (MOOCs) and our Small Private Online Course (SPOC), resulting in a blended approach [7]. We chose to use Slack for all class communication because it has shown some success in engineering education in the recent past [8], [9], and because of its popularity in industry [10] and in some educational institutions [11]. Slack Channel Structure Students were told that Slack would be the class’ primary communication tool. A dedicated Slack workspace was created for the semester. Students were allowed to join in the first week of class. The workspace channel structure looked as follows: • #general: This channel was used for class announcements, so most messages were from instructors. It was starred so that it would appear at the top of the channel list. • #news_opportunities: This channel was used for sharing news and opportunities such as contests, grants, and conferences. It was also starred. • #calendar: This channel was connected to Google Calendar to show the course schedule — most notably, deadlines. • #chapter_questions: There were several of these channels, each tied to a chapter of an online class students had to take on MOOCIBL. Students had to post questions about each chapter and collaborate to answer other students’ questions. • #presentations: This channel was used to share presentation files with classmates and instructors. It was a repository of all the presentations given by students throughout the semester (about five per group). • #instructors_only: This channel was used by the teaching staff (six people) for communication, collaboration, and coordination. • #random: This channel was used for any communication that didn’t fit in the other channels. Messages could be informal but should be relevant to the class. Methods At the end of the term, we shared a comprehensive survey with students. Ten questions were about Slack (in Appendix). This study focuses on those. Participation was voluntary: 24 out of 36 students (66.7%) took the survey. Note that not all students answered all questions. In addition, we extracted general analytics from Slack. The data are non-identifying. We analyzed the survey answers and analytics data and interpreted the former in light of the latter to draw our conclusions. Results Slack Analytics The data consist of: • 5742 messages (the free version of Slack allows up to 10,000). • 17 public channels. • 46 total members. • 6 integrated apps. • 6 GB of storage used. Workspace activity was tracked from late August to late December 2019. Note in Figure 1 the steady decline of the usage of Slack over the semester. This was expected as some students may disengage or become busy with other classes as deadlines approach. Figure 1. Weekly active members and members who posted on Slack during the semester. Beyond facilitating communication, the goal of Slack was to foster collaboration between peers. Table 1 shows the number of messages sent on Slack over the semester. Messages Sent Percentage Public Channels 574 10% Private Channels 517 9% Direct Messages 4651 81% Table 1. Messages sent over the semester: public vs. private vs. direct messages. There were eight times as many direct messages as public messages. This finding surprised teaching staff since students had been encouraged to collaborate openly, and tools had even been shared to support open exchange. The stark contrast between the figures suggests that while students want to communicate, they are less inclined to do so in public. Figure 2. Messages sent over the semester, with three peaks of activity. When looking at the number of messages sent over the semester, we noticed three peaks of activity. These roughly correspond to the start of the course, the first global evaluation, and the end-of-course submission deadline. Figure 2 suggests a correlation between evaluation periods and outstanding Slack activity. Furthermore, we can see local peaks that lined up with the meeting days (Tuesday and Thursday) of the face-to-face component of the blended class. Figure 3. Messages and reactions per channel. Next, we sorted the channels according to the number of messages posted in each. Figure 3 shows high activity in the #general, #chapter, and #presentations channels. We see that the #deliverables channel barely saw any activity. This channel was supposed to serve as a safe space for students to showcase their work — for instance, key findings — and was thought as an instrument to bolster motivation and encourage feedback. The #deliverables channel may have been misunderstood by students. Or perhaps they simply preferred sharing their progress and feedback through other avenues, such as direct messages, as suggested by Table 1. Slack can provide great insights into how the class is progressing and help determine the level of student engagement on a per chapter or per topic basis. As seen in Figure 3, in addition to messages, Slack logged the number of reactions across channels. On Slack, users may attach emojis to messages: these are called “reactions”. For instance, we can see that the #presentations channel only had one reaction; this is unsurprising considering that the channel is simply a repository for presentations. The #general channel gathered the most reactions, followed by the #chapter channels. But we sought to measure channel engagement via a combination of the number of posts and reactions. Let’s calculate channel engagement as follows: engagement = 2 ∗ reactions reactions +messages Equation 1. Custom engagement metric. This formula takes into account both messages and reactions, emphasizing the latter: given a number of messages in a channel, the more reactions they prompt, the higher the channel’s engagement level. Based on Equation 1, we can sort channels as shown in Figure 4. Figure 4. Slack channels (with 10+ messages) ordered by custom engagement metric. Besides reinforcing previous observations — that the #general and #chapter channels see significant engagement — our new metric denotes the importance of informal channels, such as the #random channel and, to a lesser extent, the #news_opportunities channel. This isn’t a surprise: informal conversation is a natural part of the soci