x 
I 
3 
2 
RATING SCALES 
(Concluded) 
Statistical and non-statistical bases for the comparison of rating methods. 
How to make a graphic rating scale. Obtaining the ratings. Statistical 
treatment of .ratings. 
CompaRIsONs of rating methods have been of two sorts, 
statistical and non-statistical (55). 
There are approximately seven statistical methods of 
comparison. 
One statistical method involves comparison of ratings and 
test scores. Rugg used this criterion (among others) in 
evaluating the Army Rating Scale. He compared ratings on 
intelligence with intelligence test scores, and found no re- 
lationship between them (156). These findings, however, 
do not argue solely against the validity of the scale, but point 
quite as definitely to the well-recognized limitations of the 
army mental alertness tests considered as measures of gen- 
eral intelligence in a broader sense. 
In the following quotation Hayes and Paterson mention 
two other methods of comparing rating scales. “The 
graphic rating method was found to be highly reliable, as 
shown by the close relationship between ratings on the same 
men by the same judge for different months, and by a close 
relationship between ratings on the same men by different 
judges” (67). Rugg also uses the latter criterion in show- 
ing the weakness of the Army Rating Scale. 
Certain precautions are necessary when such criteria are 
employed. Should a judge’s ratings vary from month to 
month or remain constant? Under certain conditions, it is 
to be expected that estimates will change from month to