CRITERIA OF VOCATIONAL SUCCESS IY
a given amount or more, and those who sold less than that
amount. Or the men may be ranked in order of their sales,
in which case there may be as many groups as there are
men.
Use of fine distinctions in vocational accomplishment is
not recommended unless the criterion is very reliable. (For
methods of ascertaining reliability, see Chapter XIII.)
It is quite as important to determine the reliability of a
criterion as the reliability of a test.
Once the criterion has been selected, care should be taken
throughout the study to keep it free from the influence of
extraneous factors. Some of the disturbing variables to be
guarded against have already been mentioned. If ratings
are used as criteria, they should be made without a knowledge
 of the tests to be used, otherwise the rater is apt to
have in mind success in these tests rather than vocational
success when he judges the workers. The criterion must
be strictly adhered to in selecting the subjects for measurement
 and in determining the abilities necessary for success
in the vocation.
The foregoing criteria are standards or measures of the
vocational success of the individual worker. In some industrial
 laboratories in Germany criteria such as these are
applied not to the workers separately, but to a whole department.
 Tests are constructed to measure the abilities
 which the investigator deems important, and without
waiting for validation, applicants who score high in these
tests are hired. When the men selected by this method
form a considerable part of the working force of the department,
 the investigator determines how much the efficiency
of the department as a whole has improved. This is measured
 by changes in the number of men required to do the
work, changes in the output of the department, reduction
in breakage, reduction in cost of training and turnover, and
so forth.
This method has little to recommend it scientifically. It
makes unwarranted assumptions regarding the ability of

=