293 REPORTS ON THE STATE OF SCIENCE, ETC. 



VOCATIONAL TESTS. 



Final Report of Committee on the reliability of the criteria used for assessing 

 the value of Vocational Tests (Prof. J. Drever, Chairman ; Mr. Eric 

 Farmer, Secretary ; Dr. W. Brown, Prof. C. Burt, Dr. J. O. Irwin, 

 Dr. C. S. Myers). 



Three types of criteria can be defined : 



(i) Objective criteria in which human judgment does not enter ; 

 (ii) Judgments by performance which are partly objective and partly 

 subjective, for they are the judgments by experts on the quality 

 of work done ; 

 (iii) Judgments of ability are subjective criteria, for they are the judg- 

 ments concerning an individual's ability. 



There are many sources of error in each of these three criteria, but no 

 one way of detecting them. A set of observations can be tested by ' Lexis' 

 theory of dispersion ' to see how far it is a valid measure of individual 

 differences. If the frequency distribution of a set of observations to be used 

 as a criterion has a low or imaginary coefficient of disturbancy, it is an 

 unsuitable measure of psychological tests. The frequency distribution of 

 all criteria should be examined by this method before they are used to 

 measure the value of psychological tests. When a criterion is shown to be 

 unreliable the cause of its unreliability can only be discovered by observation, 

 which may lead to some better criterion being formed. 



Newbold has devised a method of determining whether the frequency 

 distribution of reported accidents is due wholly to factors affecting all the 

 members of the group alike, or partly to individual differences in suscepti- 

 bility. The recorded accidents of a group cannot be used as a criterion for 

 psychological tests unless their frequency distribution is partly determined 

 by individual susceptibility. 



The reliability of judgments of performance and ability can be tested 

 by correlating independent judgments ; but judgments intercorrelated for 

 this purpose are sometimes not truly independent, in which case their 

 correlation coefficients have no value. 



A paper dealing with some of the errors in criteria and methods of 

 avoiding them has been accepted for publication in the British Journal of 

 Psychology. The Industrial Health Research Board has started an extensive 

 investigation in which the after-careers of some 2,000 apprentices will be 

 compared with their performances in scholastic and psychological tests 

 given at the time of commencing their apprenticeship. 



Both the paper and the investigation are the direct outcome of the interest 

 stimulated by the work of the Committee, which has thus served a useful 

 purpose. 



