Reliability of the Global Real-time Assessment Tool for Teaching Enhancement (G-RATE)

Despite the integral role that instructors, particularly graduate teaching assistants, play in the success of higher education, they rarely receive multidimensional feedback on their pedagogical effectiveness. In response to the need for research-based assessment tools for effective teaching and to provide feedback to graduate teaching assistants about their instructional interactions in a classroom, the Global Real-time Assessment Tool for Teaching Enhancement (G-RATE) has been developed in the context of the “How People Learn” framework. Although the G-RATE is composed of many functions, this paper presents a revised version of one component of the GRATE, the Observer function, which serves as the direct observation classroom portion in the GRATE system. Therefore, it is important to closely examine the reliability of this function. Using both pen-and-paper and video vignettes of engineering classroom occurrences, researchers report the process for determining the inter-rater (inter-observer) reliability of this function and report possible changes to the G-RATE based upon reliability measures. Future work related to the Observer function is also presented.