SELECTION 



497 



determine for which jobs rephca tests should 

 be designed, and should if possible establish 

 general principles for designing such tests. 

 Job replicas are costly and time-consuming; 

 they should be used only where indispensa- 

 ble. But, if, for any Navy job, they im- 

 prove the probability of supplying good men 

 to the Fleet, they are indispensable. 



One type of job replica, the learning 

 sample, was found to be a good predictor 

 during the w^ar for code operators and gun 

 trackers (5, 33, 59). Men who learn slowly 

 in early trials are poor prospects. Research 

 is needed to compare the selective efficiency 

 of learning samples and conventional apti- 

 tude tests. If learning samples are superior, 

 it may be necessary to redesign classification 

 procedures to allow dropping and reassign- 

 ment of men after they have started a school 

 course. 



Efficient Test Design 



The design of psychological tests has 

 rarely been studied systematically. A few 

 significant principles have emerged, but it is 

 probable that a search for general laws of 

 test design would turn up many more. As 

 examples of the sort of loiowledge to be 

 sought may be mentioned the current theory 

 that time limits should be long enough that 

 most men will finish all they can do, the de- 

 velopment of efficient formulas for item anal- 

 ysis as a technique for improving tests (9), 

 the identification of item forms which are 

 subject to response sets and should be 

 avoided (12) , and the study of the relation 

 between item difficulty and test reliability 

 (21). The last named, for example, shows 

 that a test to screen out the poorest 20 men 

 per 100 should be composed of items which 

 just 80 percent of the men can pass, rather 

 than a mixture of easy and hard items. It 

 is rules such as these that permit testers, 

 measuring a new variable, to construct a test 

 that accurately measures men in a short 

 time. 



It would be profitable to extend studies of 

 test design in several directions. One ap- 



proach is the development of tests which do 

 not depend on reading. The Bureau of 

 Naval Personnel made effective use of tests 

 in which men assembled actual gear, but 

 such tests are hard to administer. Perhaps 

 a better solution is pictorial tests (17). In 

 one extremely provocative study at Great 

 Lakes Training Center, a picture test for 

 gunners and a traditional verbal achieve- 

 ment test were given. The verbal test 

 turned out to be a measure of reading ability; 

 the picture test was a far better measure of 

 ability to operate a gun (54). Research 

 should determine the conditions under which 

 picture tests are needed, how they should be 

 designed, and what limitations they have. 



Group tests are important to the armed 

 services. Nearly every individual test 

 would be more useful if it could be given 

 accurately to groups of men. Research may 

 be able to develop a guide for converting 

 individual tests into group tests. In the ab- 

 sence of this, it is imperative that an attempt 

 be made to develop a group form of every 

 promising individual test. Examples of 

 such adaptations are the group audiometer 

 test and the projection eikonometer (23, 57, 

 pp. 77-79). 



Another type of research would stress fac- 

 tor analysis of single tests, item by item. 

 The usual method of improving a test dis- 

 cards items which differ from the remainder 

 of the test. In industrial refining, some- 

 times by-products removed from a substance 

 turn out to be more important than the main 

 material being purified. In tests, likewise, 

 study of the rejected items might call atten- 

 tion to valuable test elements now being 

 ignored . 



Instead of improving tests merely by in- 

 ternal analysis, it is possible to choose test 

 items so that each specific item is as good a 

 predictor as possible (20). This in itself 

 should improve tests for single selection tasks 

 (e.g., CIC aptitude). But if studies of job 

 criteria show certain abilities (say, mechani- 

 cal comprehension, arithmetic computation, 

 etc.) to be important in many jobs, it would 



