Easier analysis and better reporting: Modelling ordinal data in mathematics education research

This paper presents an examination of the use of Rasch modelling in a major research project, Improving Middle Years Mathematics and Science (IMYMS). The project has generated both qualitative and quantitative data, with much of the qualitative data being ordinal in nature. Reporting the results of analyses for a range of audiences necessitates careful, well-designed report formats. Some useful new report formats based on Rasch modelling—the Modified Variable Map, the Ordinal Map, the Threshold Map, and the Annotated Ordinal Map—are illustrated using data from the IMYMS project. The Rasch analysis and the derived reporting formats avoid the pitfalls that exist when working with ordinal data and provide insights into the respondents' views about their experiences in schools unavailable by other approaches. A basic requirement for any research project is the presentation of comprehensible valid and reliable results. While traditional forms of analysis can meet this requirement, other methods may be more efficacious. In this paper we present a case for the use of Rasch analysis as an approach that enables the construction of reports suitable for a range of stakeholders. We use data collected in the Improving Middle Years Mathematics and Science: The role of subject cultures in school and teacher change (IMYMS) project to support our case. The IMYMS project involves four clusters of schools from urban and rural regions of Victoria to investigate the role of mathematics and science knowledge and subject cultures in mediating change processes in the middle years of schooling. In all there are five secondary and twenty-eight primary schools involved. As is the case in many other educational research projects, the IMYMS project has provided a wealth of qualitative and quantitative data. In particular, the project researchers have used several survey instruments, and collected several sets of ordinal data. While this type of data is common in educational research, the reporting of these data often appears to ignore the mathematical properties of ordinal data, calling into question the outcomes of the research itself. It is our intention in this paper to demonstrate that the analysis of ordinal data collected using surveys and structured interviews is best achieved, through the use of Rasch modelling, and that this also provides better reporting formats. In the following examples from the IMYMS project, we report the raw ordinal data in appropriate forms, and also in transformed form as interval data using Rasch (1960) methods and a derivative, the Masters Partial Credit Model (Masters, 1988). We also present four new Rasch-based reporting formats.