All Games Are Not Created Equally: How Different Games Contribute to Learning Differently in Engineering

Reviews of game-based learning literature treat games as a unified technology whose learning contributions are comparable across cases. However, there are actually many types of games that contribute to and transform learning processes differently. This qualitative, secondary analysis of a systematic literature review catalogs six ways digital and non-digital game implementations have contributed to learning in engineering education, and classifies how radically each type of contribution has transformed learning processes in engineering classrooms. For researchers, results reinforce that contextual variables like learning objectives should be considered when studying game-based learning. For instructors, results support the merit of nondigital games as resource-effective means of transforming engineering learning processes, and suggest that teaching processes will likely change based on the game’s intended learning contribution. Introduction and Purpose In the past decade, games have developed an increasingly strong theoretical and empirical basis for effectiveness as pedagogical tools (Plass et al., 2015; Whitton, 2014). Studies have found game-based learning (GBL) to offer learning benefits in multiple disciplines, including immersive contexts to learn new languages (Peterson, 2010), authentic disciplinary problemsolving environments (e.g., Coller & Scott, 2009), and play spaces to develop social skills like teamwork (e.g., Hadley, 2014). The use of games in STEM education is of particular interest, as GBL has grown popular among mathematics instructors (Takeuchi & Vaala, 2014), has been the target of U.S. federal funding for science education (Young et al., 2012), and has taken root in engineering disciplines (Bodnar et al., 2016). Conventionally, most GBL research has focused on demonstrating the merit of games when compared to traditional teaching and learning activities (Ke, 2009). However, as GBL research continues to mature, researchers have urged the community to explore more nuanced lines of inquiry, such as evaluating the learning impact of game design components (Clark et al., 2016; Mayer, 2014) or teaching practices (Hanghøj & Brund, 2011; Kangas et al., 2016). Our study was motivated by the observation that these broader lines of inquiry continue to treat GBL as a single, unifiable pedagogy, implying that learning fostered by one GBL activity should be comparable to learning fostered by others. Studies of the effectiveness of individual game design components—known as “value-added” studies (Mayer, 2014)—seek to understand how adding a particular design component affects the educational effectiveness of a game. While this type of inquiry is effective for understanding how to modify individual games, reviews and meta-analyses examining added value game design reveal that researchers strive to find common game design components whose benefits can be generalized to game-based learning more broadly (Clark et al., 2016; Hays, 2005; Vogel et al., 2006). Similarly, studies of effective gamebased instructional practices—which we call pedagogical studies—often seek to define modes of instruction during GBL activities with minimal reference the types of games under study (e.g., Hanghøj & Brund, 2011; Kangas et al., 2016). In our (the authors’) experience, however, we have seen games that contribute to learning in several disparate ways, from fostering specific skills to offering a common prior experience to introduce a new concept. Further, while some games are relatively simple attachments to existing learning activities, others are intricate systems that help transform the learning process into something unique (Clark et al., 2016; Garris et al., 2002). Some influential authors have attempted to theoretically capture the variety of ways games can contribute to learning (e.g., Gee, 2003; Prensky, 2001), but little work has examined how current empirical applications of games for learning have actually contributed to the learning process. Understanding these differences in contributions to learning limits can benefit both game-based learning research and game-based instruction. For researchers, by understanding the ways in which different games contribute to learning differently, results from other lines of inquiry in game-based learning—such as valueadded studies and pedagogical studies—may be generalizable beyond individual games in a more meaningful fashion than generalizing broadly to all games for learning. For instructors, understanding how a particular game contributes to learning can inform how instruction should occur—e.g., what to focus on when debriefing or what kind of scaffolding is necessary to support gameplay. To investigate this problem, we elected to survey publications on games in engineering. We chose this discipline for two reasons. First, engineering—like other STEM disciplines—has seen a plethora of GBL implementations (Bodnar et al., 2016). Second, Bodnar et al. (2016) recently published a systematic literature review that comprehensively overviewed the landscape of empirical work on GBL in engineering education, and the transparency of their published methodology makes their review well-suited to a secondary analysis—i.e., using the pre-existing collection of papers to answer new research questions (Heaton, 2008). Using this systematic review for a qualitative, secondary analysis, we addressed the following research questions: 1. What have been the primary contributions of digital and non-digital games to the learning process in engineering education? 2. To what extent have digital and non-digital games transformed the engineering education learning process? We have answered these research questions by open coding for the primary learning contributions of published GBL implementations in engineering education, and by a priori coding for how transformative each game is, according to an appropriate theoretical framework. Theoretical Framework The theoretical framework we used to answer our second research question was the Replacement, Amplification, and Transformation (RAT) framework for instructional technology integration proposed by Hughes et al. (2006). Drawing from findings on technology integration in prior literature as well as observations from the lead author’s research, the RAT framework categorizes technology use—in this case, game implementation—based on the degree to which it transforms or enhances learning tasks. The framework has three such categories, in increasing order of enhancement: 1. Replacement – The game does not enhance or change learning tasks in any meaningful way, serving “merely as a different means to the same instructional end” (Hughes et al., 2006, p. 1617). 2. Amplification – The game does not change the learning tasks, but enhances them in ways not feasible without the game. Enhancements can provide learning aids, such as contextual help systems and visualizations; or can increase learning productivity, such as through automation of calculation or assessment. 3. Transformation – The game allows for the inclusion of learning tasks that would not be feasible otherwise. Transformation can involve the introduction of new subject matter, teaching practices, or learning processes. Several studies have demonstrated that the RAT framework is useful in categorizing instructional technology integrations with respect to how technologies modify learning tasks (e.g., Kimmons et al., 2015; Smidt et al., 2012). In this study, we applied the RAT framework to games, which we consider to be instructional technologies because they are a form of media used to complement instruction (Gagné, 2013). Specifically, we used the RAT framework to categorize how transformative each game was, compared to typical active learning tasks in engineering classrooms, such as quizzes, labs, design projects, closed-ended problem-solving tasks, case studies, and programming tasks.