Premium
Ranking or rating? Some data and their implications for the measurement of evaluative response
Author(s) -
Russell P. A.,
Gray C. D.
Publication year - 1994
Publication title -
british journal of psychology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.536
H-Index - 92
eISSN - 2044-8295
pISSN - 0007-1269
DOI - 10.1111/j.2044-8295.1994.tb02509.x
Subject(s) - ranking (information retrieval) , psychology , rating scale , painting , point (geometry) , subject (documents) , statistics , social psychology , developmental psychology , mathematics , artificial intelligence , computer science , art , geometry , library science , visual arts
Two groups of 80 subjects rated 15 paintings on a five‐point pleasingness scale in accordance with a ‘relative’ instruction, requiring them to compare the paintings with one another, or an ‘absolute’ instruction, requiring them to compare the paintings with other, familiar ‘anchor’ paintings. Each subject also ranked the paintings on pleasingness, either before or after doing the ratings. Within‐subject correlations between ranks and ratings revealed that, for all but a small minority of subjects, there was a moderate to high level of correspondence between the two measures. There was a tendency for those subjects who had shown good correspondence between their ranks and their ratings to show a greater spread in their ratings. The relative instruction also produced a greater spread in subjects' ratings compared with the ratings obtained under the absolute instruction. The means of the ranks and the ratings that the subjects assigned to the individual paintings were highly correlated in both groups. The results are discussed in terms of the respective merits of ranking and rating procedures, and it is concluded that, where practical and statistical considerations permit, and where the items are highly discriminable, a ranking procedure has advantages. Where ranking is precluded, however, a rating procedure with ‘relative’ instructions offers some of the same desirable characteristics.