Assessing Cognitive Development and Motivation with the Online Watershed Learning System (OWLS)

A recent report on Challenges and Opportunities in the Hydrologic Sciences by the National Academy of Sciences states that the solutions to the complex water-related challenges facing society today begin with education. The Learning Enhanced Watershed Assessment System (LEWAS) is a real-time watershed monitoring lab that seeks to address these complexwater related challenges by improving water-related education at the community college and four year university levels. The Online Watershed Learning System (OWLS), the data sharing and visualization component of the LEWAS, is an environmental exploration tool that gives users access to historical and live LEWAS data, watershed-specific case studies, and virtual tours of the LEWAS watershed. By using an HTML5-driven web interface, the OWLS interactively delivers integrated live and/or historical remote system data (visual, environmental, geographical, etc.) to end users regardless of the hardware (desktop, laptop, tablet, smartphone, etc.) and software (Windows, Linux, iOS, Android, etc.) platforms of their choice. We have built upon a prior study that used the expectancy-value theory of motivation to show that exposure to live watershed data via the LEWAS increased students’ levels of motivation. A pilot test of the OWLS has demonstrated positive learning gains in engineering seniors and was overwhelmingly viewed by students as having helped them learn hydrology concepts. The pilot test also revealed the strengths of the OWLS to be anywhere, anytime access to live system data and interactive graphical representations of the data. Using the framework of situated learning, the current research implements the OWLS as a remote lab for both freshmen community college students in general engineering courses as well as senior university students in a hydrology course. We seek to determine: (i) how the OWLS influences student learning with respect to course learning objectives, and (ii) how the use of OWLS in engineering courses impacts motivation in students. The assessment follows an experimental design with preand post-test questions that include both Likert-style motivation questions and concept inventory-style cognitive learning questions that have been developed by content experts for each course level and are scaled using Bloom’s Revised Cognitive Taxonomy. Results from fall 2014 freshmen course are analyzed and presented and results from both levels in the spring 2015 semester will be included in the presentation. 1.0 Introduction In 2008, the U.S. National Academy of Engineering (NAE) announced 14 Grand Challenges in engineering that are awaiting solutions in the 21st century. This list includes the challenge to “Provide Access to Clean Water” 1 . Water is the critical resource for supplying food and energy, safeguarding human health and maintaining national security. Increasing pressures for water demand worldwide present challenges to scientists and engineers to attain sustainable management of water resources. A recent United Nations report projects that virtually every nation will face a water supply problem within the next 8 years; currently more than a billion people have little access to clean drinking water, and 2 billion live in conditions of water scarcity 2 . To address these critical issues, the NAE’s “The Engineer of 2020” highlights the need for implementing ecologically sustainable practices to preserve the environment for future generations. Further, the report emphasizes that water supplies will affect the future of the world’s economy and stability 3 . As a result, the NAE warns that unless better ways to protect and improve water supplies are found, the future looks dire for billions of people 4 . To prepare for these challenges, the educational system must teach our youth about critical hydrology related issues and train them as future professionals who are capable of developing appropriate solutions. Two of the greatest challenges facing hydrology education in the 21 st century include providing student-centered activities and field experiences in the classroom 5 and replacing historical stationary data with real-time, dynamic, and temporally and spatially variable hydrologic systems 6-7 . Replacing traditional teaching methods with studentcentered experiences that incorporate non-stationary data will require advances in classroom tools and teaching methods that capture the attention of students through an active learning experience. Incorporating student-centered learning through virtual and remote laboratory experiences 8-10 that situate users in a remote hydrologic site is a common method for achieving this goal 11-12 . This study investigates such an educational tool by determining the impact of the Online Watershed Learning System (OWLS) on student learning and motivation in university and community college classrooms. The OWLS is an online tool that broadcasts real-time, highfrequency environmental data (flow, water quality and weather) from the Learning Enhanced Watershed Assessment System (LEWAS) located in a watershed on the campus of Virginia Tech. The LEWAS is an environmental monitoring lab that collects water quality, flow and weather data in high-frequency (1-3 minute) intervals in a stream that drains a 2.8 km 2 urban watershed. The OWLS allows users to remotely explore the watershed through access to realtime data, geographic watershed tours, and watershed-specific case studies. Students use the OWLS to participate in hands-on remote lab activities that virtually situate the students from the classroom into the field. This study seeks to enhance student learning and motivation by incorporating the OWLS activities into the curriculum to engage students in active learning while supporting course objectives in university and community college classrooms. This study focuses on how the OWLS influences student learning with respect to course objectives mapped to learning outcomes as defined by ABET a-k criteria 13 and influences student motivation using the MUSIC model 14 to address the following research questions: (1) How does the OWLS influence engineering students’ abilities with respect to course learning objectives?; (2) How does engineering students’ use of the OWLS relate to their motivation levels?; and (3) How does student learning and motivation vary across institutional contexts (i.e., university vs community college) in students that are exposed to the OWLS? The outcomes of this study will ultimately result in a greater understanding of how remote lab technologies can bring field experiences into the classroom via online access to dynamic real-world data to enhance student learning and motivation in hydrology education, thus addressing one of the grand challenges facing hydrology education in the 21 st century of providing real-time, dynamic, and temporally variable data 6 . This study also provides insights into how technologies such as the OWLS can be used to support classroom learning objectives that map to ABET criteria. The focus of this paper is on the background, theoretical framework, and methodological approach of the study, with results presented from a pilot test conducted in the community college courses during the fall 2014 semester. Specific details on the design of the LEWAS system and the OWLS and results from the first pilot test in the hydrology course during the spring 2014 semester can be found in previous publications 15-16 . The presentation associated with this paper will also contain the final results from the OWLS implementation in the spring 2015 university and community college courses. 2.0 Background An advantage of the LEWAS is the ability to collect, store, and transmit data in real-time, which can be displayed through an environmental virtual or remote lab, such as the OWLS, where students can explore the environment, case studies, and live data. Virtual labs are software that simulate the real environment, whereas remote labs are labs where experiments are conducted remotely across the Internet. Virtual labs have been shown to be effective in improving student understanding of important engineering concepts 17-19 . For example, researchers at UCLA found that students perceived learning gains when using the Interactive Site Investigation Software (ISIS) to perform virtual field work such as constructing wells, collecting groundwater samples, submitting samples for laboratory testing, and executing hydraulic transport experiments 10 . Applications of remote labs in engineering education have also been shown to improve student understanding of engineering concepts 20-21 and are comparable to hands-on labs 8-9,22 . For example, researchers at Rutgers University found that there was no difference in educational outcomes between students who participated in a remote lab versus an in-person lab 9 . The OWLS uses components of both virtual labs (students can virtually explore a simulated environment through geographic representations) and remote labs (students can choose which parameters they want to measure) to give users a unique educational experience. Pilot tests of the OWLS have been implemented in two freshman level introduction to engineering courses at Virginia Western Community College and into a senior-level hydrology course at Virginia Tech during the 2014 school year. The OWLS was implemented into each course using classroom modules that are based upon previous work integrating real-time, highfrequency LEWAS data into the classroom 12, 23-25 . These previous studies found that students who were exposed to real-time environmental data had improved levels of motivation 23 and that students who participated in LEWAS-based modules that used high-frequency data experienced significant learning gains 16 . Using the OWLS, this study seeks to build upon previous work by providing an interactive online watershed education tool that gives students access to real-time, highfrequency environmen