Use of Software Agent-Monitored Tutorials to Guide Student Learning in Computer-Aided Design, Analysis and Mathematics Projects

Internet chat-based tutorials are being developed for integrating computer modeling and design and mathematics skills into mechanical engineering undergraduate and middle school outreach programs. In modeling and design projects, tutorials help students navigate complicated software interfaces while teaching fundamental concepts through dynamic dialogues between tutorial agents and student user groups. In a typical assignment, students are asked to perform a design or modeling task that includes the use of software such as a commercial finite element code or specially designed educational software. Students work in teams, but team members are distributed within a room or between remote sites, linked by a text interface. As students collaborate electronically, an intelligent agent monitors their interactions and interjects questions or comments in response to the use of key phrases, or due to other triggers. This platform is also being adapted to the collaborative teaching of mathematics skills in engineering applications. In all projects, agent-monitored tutorials are being used to help automate collaborative learning experiences and to study how students can effectively interact with each other and with the software agents. In undergraduate projects, fundamental knowledge and intuition in interpreting results are emphasized. In outreach efforts, participants are led to consider how their work relates to the broad mechanical engineering profession. Introduction Over the past decade, the landscape in which engineering is practiced has been radically altered by two trends. First, there have been far-reaching technological advances in computing, information-sharing, and automated manufacturing. Second, there have been increasingly strong economic and market pressures for shorter product development cycles and internationally distributed team-based product design and manufacturing. Many of the same technological advances that have transformed engineering practice also have the potential to transform engineering education. For instance, it is widely recognized that guided use of sophisticated simulation software can enable exploratory and inquiry-based modes of learning . If these capabilities are exploited effectively, then integration of simulation-based projects can efficiently increase student learning and understanding. In a related way, internet-based instruction coupled with on-line student collaboration, and the real-time transfer of numerical product designs, numerical simulations, images and product prototypes offers a wide range of possibilities for student learning. Instructors are no longer limited to the use of traditional, on-campus, classroom-based lectures. Recognizing this, we have been engaged in ongoing research both to transform freshman level engineering instruction 10 as well as sophomore level thermodynamics instruction 16, 17, 2, 6 using collaborative project-based learning modules. These modules not only exploit on-line collaboration capabilities, but also employ the use of programmed intelligent software agents, which monitor student interactions and then interject comments and questions designed to teach students directly and/or increase student collaborative learning by teaching each other. Early results have showed improvements in learning of over a letter grade in magnitude for students P ge 22597.2 working with a human partner and a computer agent in comparison with students working alone without the agent. In a series of controlled classroom studies we have continued to improve learning effectiveness through changes in the computer agent design. Important innovations include offering students control over the timing of feedback, using social strategies motivated by the field of collaborative group work , and developing agents that demonstrate alignment with student goals. The underlying thesis of this research is that offering a dynamic self-paced learning environment for student use outside of the lecture room is the best practical means for integrating sophisticated design and analysis experiences into undergraduate engineering curricula. Furthermore, the machine-monitored internet chat-based tutorial environment we use to achieve this goal offers an excellent opportunity for automating and invigorating K-12 outreach efforts and for tying them naturally to more sophisticated undergraduate-level instruction. The foundation of our approach consists of two pillars: 1) self-paced web tutorials guiding students through software use and 2) dynamic, dialogue-based tutorial interfaces which engage students in interpreting simulation results they create. The use of self-paced web tutorials as a means of efficiently integrating complex software package use into undergraduate curricula has been the subject of a long-term effort at Carnegie Mellon . The integration of an agentmonitored dialogue-based interface into software instruction represents a substantial enhancement to this approach. As we deploy dialogue-based tutorials, we are not simply using them to enhance software instruction. We are also studying how students learn and teach each other in a tutorial-guided dynamic chat environment involving students in one or multiple groups. Our initial focus has been a first-year introductory and second-year thermodynamics mechanical engineering courses; however, our goal is to establish a template for a new approach to design and analysis instruction throughout our curriculum. Furthermore, we are also applying our agent-monitored dialoguebased tutorial platform to the task of middle school student outreach. Tutor Architecture and Interface In this research, we are adapting a prototype architecture we have developed for supporting student interactions in a broad range of activities . This unique, automated, collaborative learning platform naturally exploits chat room-style communications that are ubiquitous on the internet, and also students’ comfort and curiosity with that environment. In our prior work using a similar environment for middle school math instruction, students found the collaborative problem solving environment highly engaging. Some students commented that the collaborative environment was “way more fun” than their typical computer lab activities, and that they were disappointed when the 45 minutes lab session was over. The idea behind its design is for a filter to process the text from an ongoing discussion as it is happening, and to build an internal model of how the conversation is progressing. Using this model, it is possible to determine where the most strategic opportunities for supporting learning exist. Figure 1 shows an overview of the architecture used to develop our prototype infrastructure. This architecture is meant to allow context-sensitive support for collaborative learning and reflection not only to be triggered based on what is happening in the discussion, but P ge 22597.3 Conversational Agents e.g. TuTalk Participants (Students) Filters Event Notifications Internal Triggers e.g. Topic Shift External Triggers e.g. Timer Knowledge Sources e.g. Recipes Communications for it to do so with awareness of how it is affecting the state of the conversation through its continuous monitoring. Thus, if an intervention is triggered erroneously and ends up having a negative effect on the collaboration, we can detect and correct that. In this way, we minimize the risk of misdiagnosing the state of the collaboration. As displayed in Figure 1, all interface events resulting from student contributions to the chat interface and to a shared problem solving or simulation space are sent to the Filters module. Its purpose is to identify significant events in this stream that it then reflects back to the interfaces of the students. It also uses these identified events to update its internal state. Other triggers such as timers that keep track of time elapsed since the beginning of the session or since the last significant contribution of each student are also used to manipulate the Filter module’s internal state. The internal state then is used to select strategies for selecting dialogue agents to participate in the chat session in order to offer support in the form of interactive directed lines of reasoning. In our prior experiments we have used different kinds of triggers including topic-based filters, time-outs, interface actions, and conversational actions that are indicative of the degree of engagement of the students in the discussion. Our generic architecture is meant to be easily extended to work with other types of triggers such as cues from other modalities like speech, hand sketches, etc. We continue to improve the architecture to provide richer communication and modularization. Conversational agents meant to offer support in the midst of collaborative learning interactions can be authored with the TuTalk dialogue agent authoring system . As displayed in Figure 1, when the Filters module sends a notification to the Conversational Agents module to trigger a particular cognitive support agent, the scheduled TuTalk agent is appended to a queue of TuTalk Agents, which are then launched in turn at appropriate breaks in problem solving behavior. Figure 1: Architecture of the collaborative problem-solving interface with conversational agents