Peer Review For Online Learning Objects Via Merlot

With the rapid advances in computer technology and software over the past 10 years, it has become increasingly attractive from a pedagogical viewpoint to develop computer-based simulations and tutorials to allow our students to explore and learn chemical engineering principles. However, development of such “learning objects” can be given short shrift when professional contributions are evaluated, partly because they typically have little impact outside the class for which they are developed, and partly because professional review committees seldom feel they have a basis for evaluating the quality of the contribution. In contrast, professional review committees are able to rely on peer review of proposals and journal publications when evaluating the quality of a faculty member’s contributions in technical research. For faculty who invest time and effort to develop learning objects because they feel it is “the right thing to do”, it would be pleasant to have a mechanism for broad dissemination and peer review of these contributions. This is one step to increase the respect for teaching as research and research on learning in engineering programs. The Multimedia Educational Resource for Learning and Online Teaching (MERLOT, www.merlot.org) is a database of educational resources, primarily for post-secondary education. It includes both discipline-specific resources (e.g., demonstrations, tutorials, on-line experiments, course notes) and more general resources for educational research and improvement (e.g., guidelines for writing and assessing student learning outcomes). Although other databases exist, MERLOT is unusual because it includes a system for peer review. Editorial boards assign objects already in the database to reviewers with relevant technical expertise. Reviewers’ comments on technical content, ease of use, and educational potential are then displayed in the database along with the link to the learning object as well as suggestions for how to incorporate the learning object into a course. The MERLOT engineering editorial board is actively seeking chemical engineering learning objects to be linked to the database, chemical engineers to share strategies for using these learning objects, and chemical engineers to act as peer reviewers. This presentation will provide an overview of the chemical engineering learning objects currently linked to the MERLOT database. However, it will focus on the MERLOT peer review process, and the information that it generates for users of the learning object and for reviewers of the professional contribution. What is MERLOT? The Multimedia Educational Resource for Learning and Online Teaching (MERLOT) database is a catalog of on-line tools for teaching and learning. The tools, or “learning objects”, are owned by their authors and do not reside on MERLOT computers. MERLOT catalogs these learning objects by subject area, provides links to them, and provides descriptions and peer reviews that indicate the learning objectives addressed by the object and document the Page 10994.1 “Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition Copyright ©2005, American Society for Engineering Education” contribution of a learning object to the scholarship of teaching. The MERLOT homepage (Figure 1) url is www.merlot.org. Figure 1. The MERLOT homepage Users can browse by subject or search by key word. Pedagogical resources useful in any discipline are found under “Education”. Resources for engineering course content are found under “Science and Technology”. One goal of the MERLOT engineering editorial board is to build the collection and the traffic in engineering so that engineering will become a member of the top-level subject list, not a category under Science and Technology. Anyone can browse the material in MERLOT, follow the links, and use them. Chemical engineering educators are encouraged to become members (it’s free) and help build the MERLOT community. Membership does not require you to contribute or review objects or otherwise serve. Learning objects enter MERLOT upon being added by a MERLOT member. Any member can add a learning object by clicking on “Contribute Material”. MERLOT is a catalog, not a repository. The author of a learning object retains full copyright and full control. MERLOT members are encouraged to add any useful online learning object, not just objects they author. When you add a learning object, you provide a brief description of the object and identify the subject area(s) under which it should be filed. The object will appear in the database immediately; peer review follows. P ge 10994.2 “Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition Copyright ©2005, American Society for Engineering Education” Each major discipline areas in MERLOT has an editorial board. (Valerie Young is a member of the MERLOT Engineering Editorial Board.) Editorial Boards periodically triage new learning objects and identify promising ones for peer review. A peer-review team consists of two educators in the discipline who rate the learning object for Content Quality, Potential Effectiveness as a Teaching Tool, and Ease of Use. The peer-review team also summarizes the learning goals and target student population for the learning object, and suggests ways in which it might be used. The MERLOT peer reviews are true reviews, separate from the “amazon.com”-style user comments that may also appear for an object. MERLOT is unique because it (1) covers a wide range of disciplines, (2) is focused at the university level, and (3) includes peer review and other ancillary documentation to assist educators in using the learning objects and in obtaining credit for developing them. Computer Aids for Chemical Engineering (CACHE), is a not-for-profit organization whose purpose is to promote cooperation among universities, industry and government in the development and distribution of computer-related and/or technology-based educational aids for the chemical engineering profession. (The CACHE collection is catalogued in MERLOT.) However, CACHE does not grow “from the grass roots” as MERLOT does. CACHE focuses on chemical engineering, while MERLOT spans a variety of disciplines. Also, links in MERLOT are regularly verified and dead links removed. One can also uncover a variety of learning objects using a standard search engine such as “Google”. However, these will lack the peer reviews and suggested assignments that MERLOT provides. The Peer Review Advantage to Learning Object Developers Promotion and tenure committees measure research “quality” in a variety of ways: presentations delivered, publications in peer-reviewed journals, grant proposals funded, comments by reviewers of proposals and publications, graduate degrees completed, reference letters from established researchers in the field. However, in many engineering schools, measurement of teaching “quality” relies heavily or exclusively on student evaluations. Engineering faculty often develop learning objects for personal satisfaction, not because they expect this contribution to be recognized in the promotion and tenure process. The development of learning objects is undervalued in professional evaluations at least partly because committees are unsure how to evaluate their quality, contribution to student learning in the author’s course or in the broader discipline community, or potential to enhance the reputation of the author and the institution. For the educator who develops learning objects, MERLOT offers two advantages. First, it makes your work widely accessible without your losing control over it or having to redevelop it to meet externally-imposed standards. Second, it provides formal documentation of the quality of your work and its potential impact on student learning in and beyond your individual course. Current MERLOT Chemical Engineering Content As of January 2005, MERLOT catalogued 326 engineering learning objects, 24 in chemical engineering. These include simulations, virtual laboratories, tutorials, calculation tools, and other collections of links and objects. For example, “McCabe Thiele Method” is a tutorial by Venkat R. Bhethanabotla of the University of Southern Florida that includes text background on distillation, describes and illustrates the McCabe-Thiele method, and includes a self-test. “Chemical Reactivity Worksheet”, produced by the National Oceanic and Atmospheric P ge 10994.3 “Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition Copyright ©2005, American Society for Engineering Education” Administration, is a tool for projects in a safety or design course which allows the user to evaluate the risk of mixing any of more than 6000 hazardous substances with air, water, or other chemicals. “Process Dynamics Online Laboratory” and “Control Systems Lab Online” developed by Jim Henry at the University of Tennessee Chattanooga allow students to run actual experiments remotely. In addition, chemical engineering educators find a variety of useful learning objects in the chemistry, mechanical engineering, and materials engineering catalogs, and pedagogical tools in the education catalog. Some objects are cross-listed in multiple subject areas. Chemical engineering educators are encouraged to add any objects they have found useful. If you are reluctant for some reason to join MERLOT, e-mail the url for the object to Valerie Young (youngv@ohio.edu) and I will do it. Example Peer-Reviewed Learning Object When you click on the listing for an object in MERLOT, the “detail view” appears. Figure 2 shows the detail view for “Virtual Tour of a Steam Turbine Cogeneration Power Plant”, by Carl Lira of Michigan State University. Clicking the links at the top of the page will allow you to view the Peer Reviews and suggested Assignments for this object. Figure 2. A peer-reviewed learning object “detail vie