Impact Of A Nsf Ate Funded High School Science And Technology Outreach Program: Evaluation Of H.S.T.I. Materials

The National Science Education Standards state that "any presentation of science without developing an understanding of technology would portray an inaccurate picture of science." It further notes; "High school students do not distinguish between the roles of science and technology". Today’s high school students are exposed to an ever-increasing amount of high technology that impacts their everyday lives. Still, the number of students that possess knowledge or understanding of the underlying principles, or interest in the development and/or manufacturing background of these technologies is small. This lack of knowledge and interest has contributed to U.S. firms in the U.S. high-technology sector looking outside the country in order to find workers with the right skills. With these factors in mind, the High School Technology Initiative (HSTI) project was launched and funded by NSF-ATE. HSTI offers materials that provide science and math content designed to connect students and teachers to today’s technologies. HSTI materials are Modules and Module Usage Guides (MUG) developed for science, mathematics and technology teachers. The Modules are topic based, supplementary teaching tools, designed to connect science and technology. Module Usage Guide (MUG) materials are Workshops and Short Courses. The MUG Workshop is designed to familiarize the teacher with the structure of the HSTI modules and offer suggestions for classroom integration. The Short Courses are the professional development portion of the MUG. They are classroom-based, in-depth training on the technologies associated with the science presented in the respective module. In the past two years, 180 teachers have accepted HSTI modules impacting nearly 20,000 students. During the HSTI project, we have used mixed methods of data collection, including online teacher surveys regarding the modules, post professional development surveys, student impact prereporting by the teachers, and direct observations. The purpose of our sampling was to create an accurate composite picture of the teachers and students exposed to the HSTI materials and collect data to improve our product and process. Our strategy was to primarily solicit responses from the teachers, due to the difficulty in surveying students directly. This poster presentation will offer an overview of HSTI materials, discuss the evaluation methodology, and present evaluation results. P ge 10716.2 Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition Copyright © 2005, American Society for Engineering Education. Introduction The National Science Education Standards state that "any presentation of science without developing an understanding of technology would portray an inaccurate picture of science." It further notes; "High school students do not distinguish between the roles of science and technology" [1]. Today’s high school students are exposed to an ever-increasing amount of high technology that impacts their everyday lives. Still, the number of students that possess knowledge or understanding of the underlying principles, or interest in the development and/or manufacturing background of these technologies is small [2]. This lack of knowledge and interest has contributed to U.S. firms in the U.S. high-technology sector looking outside the country in order to find workers with the right skills [3]. With these factors in mind, the High School Technology Initiative (HSTI) project was launched and a proposal was submitted and funded (DUE 02-02373) by NSF-ATE. Project funds support the construction of three High School Technology Initiative (HSTI) modules. Two of the three modules, "The Problem Solving" Module and "The Atom" Module are completed, beta tested, and are in use in schools in eleven states (Arizona, California, Connecticut, Colorado, Florida, Idaho, New York, Oregon, Tennessee, Texas, and New Mexico). The third module, "Fields and Waves", is on schedule for completion by June 2005. HSTI modules have proven to be an effective way for teachers to deliver science and math related content to students. They also present technology content lessons within the time frame allocated by the instructors’ selected presentation format. The modules provide teachers multimedia presentation options for each lesson. Each module contains presentations, worksheets, in or outside class activities, as well as quizzes and tests. In summary, each HSTI module offers high school science and mathematics teachers curriculum content material that emphasizes technology as it relates to science and mathematics, as well as providing technology based multimedia options as delivery vehicles. A HSTI module is developed not only with the high school science educator presentation in mind, but with science and mathematics content created as the high school curriculum has dictated. The HSTI development team consists of high school science teachers, university and community college professors, engineering Ph.D. students, and multimedia professionals. The typical mode of operation for the team is to review the topics currently presented specifically with the high school teachers. These discussions lead to the module's science and mathematics content as well as identification of the science, mathematics and/or technology standard that the material will address. Ultimately, the science and mathematics objectives are blended with technology and engineering related examples, to produce a module that integrates technology with the specific science principles and mathematics skills required of the lesson plan, which the module is designed to support. The HSTI team believes that HSTI modules can be used to engender an interest in pursuing technology, engineering, or science related careers by providing students with connections between everyday examples of technology and their underlying science as part of the normal state mandated science instruction. Also, HSTI modules augment the high school science curriculum with technology content material that is standards based, has a positive impact on students, is attractive to the science and mathematics teacher, and is an effective, efficient and P ge 10716.3 Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition Copyright © 2005, American Society for Engineering Education. appropriate approach to connect technology to fundamental science concepts and mathematics principles. Evaluation of HSTI Module Usage The major purpose of the evaluations done thus far, have been to study the acceptance, usability, and impact of HSTI materials (Modules and Module Usage Guides). Further, these evaluations were intended to provide objective, unbiased feedback to the HSTI team for the purpose of module refinement and guidance on development of future materials. The following questions are central to the evaluation within the context and goals of each module and the overarching mission of the entire HSTI project. • To what extent are students being exposed to HSTI materials? • What materials are the teachers comfortable in using? • What barriers to meeting student learning can be identified? • To what extent are the stated goals and objectives of the project being met? • Are the professional development opportunities offered by the MUG’s beneficial? During the HSTI project, we have used mixed methods of data collection. This include online teacher surveys regarding the modules, post professional development surveys, student impact prereporting by the teachers, and direct observations. These measures address the acceptability of a variety of school-based instructional and intervention strategies with teachers, parents, students, and consultants, and have been used in both analog and naturalistic settings. Acceptability measures contain common item stems (relating to, e.g., like, understand, easy to use, effective with students, use again, recommend to others), with varying rating targets (instructional or treatment strategies), and a common 5-point Likert response format (1 = strongly disagree, 5 = strongly agree). Measurement reliability is consistently high, as is construct validity with factor analyses typically yielding a single general factor of acceptability [4]. The purpose of our sampling was to create an accurate composite picture of the teachers and students exposed to the HSTI materials, and collect data to improve our product and process. Our strategy was to primarily solicit responses from the teachers, due to the difficulty in surveying students directly. Listed below are surveys that were administered: • PSM & Atom Promissory Note Questionnaire (2002-03) • PSM & Atom MUG Workshop Survey (2001-03) • PSM MUG Short Course Survey (2002-03) • Atom MUG Short Course Survey (2003-04) • PSM & Atom Online Module Evaluation Survey (2004) The Problem Solving Module (PSM) and Atom Module Promissory Note is the agreement between HSTI and the teachers planning on using the modules. Teachers are asked to sign the following statement upon receipt of a HSTI module: “By accepting this CD and/or video, I agree that I will use the material in my class(es) listed below and will provide information to HSTI for their internal use and for use in the NSF grant evaluation process.” They are then asked to participate in module evaluation surveys, what course they will use the module in, and how many students will be exposed to the HSTI module materials. The questionnaire data indicates P ge 10716.4 Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition Copyright © 2005, American Society for Engineering Education. that 180 teachers want to use the module in their classes exposing 20,000 students to the HSTI module materials. These teachers represent schools in 11 states. The PSM & Atom MUG Workshop is designed to familiarize the teacher with the structure of the HSTI modules and offer suggestion for cl