302 MEMOIRS NATIONAL ACADEMY OF SCIENCES. [Vol.xv, 



5. Information. — List of 40 items. Time limit, 4 minutes. Length of line approximately 

 12 words, so that each statement may occupy single line, with space for four alternate responses. 

 Responses to be made by underscoring appropriate word. 



6. Synonym-antonym. — Ten sets of 40 items each to be prepared. Time limit, 2 minutes. 

 All the items in a single column. Response indicated by underlining "same" or "opposite." 

 A dotted line should connect the second word of each pair with the response words. 



7. Practical judgment. — Ten to 20 items. Time limit, 2 minutes. Alternate responses 

 to be given, one of which is to be checked. The number of alternating responses and the method 

 of checking them is to be determined by those delegated to work out the tentative series. The 

 check preferably is to be brought, if possible, within two lines. The language must be simple 

 and direct. 



8. Number series completion. — Ten items, introducing in all four different principles. Time 

 limit, 4 minutes. 



9. Analogies. — Forty items, with a time limit of 4 minutes. There should be four responses 

 from which to choose, only one of which is correct. At Whipple's suggestion it was decided to 

 use for the three incorrect responses words which would naturally be given as a reaction in a free 

 association test when the third term of the analogies was used as the stimulus word. 



10. Number comparison. — Twenty columns of numbers, consisting of six or seven numbers 

 each. The subject is to underscore the smallest number in each column. (This was later 

 changed to the smallest and largest numbers in each column, in order to increase the test's 

 demands upon the higher intellectual processes.) The numbers are to consist of two to four 

 digits. The first digits in any one column are to be neither wholly the same nor wholly different. 

 The smallest have at least one duplicate. Time limit, 2 minutes. 



Since it was agreed to have 10 equivalent forms of each test as a safeguard against coaching, 

 the question arose as to the best method of guaranteeing the equality of the different forms as 

 regards difficulty. A solution proposed by Wells was adopted for all of the tests except that 

 of oral directions, namely, the principle of random selection. This principle may be illustrated 

 as follows: If it is desired to secure 10 different sets of the analogies test, each set containing 

 40 items, the method would be to collect 400 such items, shuffle them thoroughly, and select the 

 items for the 10 different sets by drawing. It will later be seen how satisfactorily this method 

 worked. 



On June 1 the tests selected for group examination were assigned to various members of the 

 committee to prepare the items for the equivalent forms. This work continued until June 4. 

 The assignments for preparing the test material were as follows: 



Oral directions to Whipple. 



Memory for digits to Wells. 



Disarranged sentences to Haines and Goddard. 



Arithmetical reasoning to Bingham. 



Information to Wells. 



Synonym-antonym to Terman. 



Practical judgment to Haines and Goddard. 



Number series completion to Wells. 



Analogies to Terman. 



Number comparison to Wells. 



It should be stated, however, that no one's contribution was limited to his specific assign- 

 ment. Every member of the committee contributed helpful suggestions on practically every 

 test. When tentative fists had been made out they were presented for the criticism of the 

 group. It is impossible to apportion credit justly, for the work was primarily an example of 

 what Royce has called "the fecundity of aggregation." 



Wells's conspicuous share in this work was in part due to the fact that he had brought with 

 him a large amount of material on memory for digits and number comparison which could be 

 utilized without change. This material had already been arranged by the method of chance 



