544 



SCIENCE 



[Vol. LVI, No. 1454 



As indicated on the curve and shown on the 

 tables (which it seems unnecessary in this place 

 to print) two and only two psychologists were 

 included among 'the fifty leading psychologists 

 by unanimous vote of their eighty-two col- 

 leagues. The psychologist at the bottom of 

 the fifty received 35 votes; three were given 

 for the psychologist who stands last in the 

 hundred. Sixty-two of the 82 voting include 

 among the five whose work has been most im- 

 portant the two psychologists who received a 

 unanimous vote for inclusion among the fifty 

 and 20 do not. Such differences in judgment 

 are legitimate and significant. Thus the psy- 

 chologist placed eleventh is held by 22 of his 

 colleagues to belong to the first five and by 21 

 not to belong to the group of 50, and similar 

 conditions obtained for the one placed twen- 

 tieth. The names could 'be guessed by one 

 familiar with the situation. They are men of 

 distinction whose more important work is by 

 some judged not to fall within the field of 

 psychology. 



The attitude of those voting is of scientific 

 significance, for it measures the validity of 

 judgments. If we assume the average judg- 

 ment of the 82 psychologists to be nearly cor- 

 rect, the departure fix)m this average measures 

 the competence of the individuals to form such 

 judgments. There do not appear to be group 

 differences dependent on distinction or age, 



ranging from 175 chemists to 20 anthropologists. 

 The returns, however, were most complete for 

 psychology, the writer feeing personally acquaint- 

 ed with nearly all psychologists and a second 

 request having been sent to those who did not 

 reply to the first. There were in all some 130,000 

 votes to be collected, counted and tabulated. For 

 the treatment of this material I am mainly 

 indebted to my daughter. Miss Psyche Cattell. 

 Dr. Dean E. Brimhall and Dr. Alexander "Wein- 

 stein have also assisted in the revision of the 

 material and in the computations. 



I am under very great obligations to Professor 

 Raymond L. Pearl,' of the Johns Hopkins Uni- 

 versity, and to Professor H. L. Eietz, of the State 

 University of Iowa, for their kindness in reading 

 the manuscript of the paper and for the useful 

 suggestions that they have made. This acknowl- 

 edgment should not, of course, be construed as 

 involving responsibility on their part. 



but individuals, as shown in the previous study, 

 differ in the ratio of about two to one. 

 There "will be an extraordinary change in 

 our attitude toward political, social and busi- 

 ness problems when we learn to look upon our 

 obsei'vations, recollections, beliefs and judg- 

 ments objectively, measuring the probability of 

 their correctness and assigning probable errors 

 to them. 



The top curve represents the sum of the two 

 votes and the order there given is the one used. 

 A different weighting of the two votes would 

 not eonsideraibiy affect 'the order. The vote for 

 the five leading psychologists in the main dis- 

 criminates only the positions of the men in the 

 upper quartile. In place of the double vote 

 for the five and the fifty per cent, of the 100 

 psychologists whose work has the most merit, a 

 satisfactory distribution might be obtained by 

 a vote for 25, or one fourth of the whole num- 

 ber. In view of the constant use of votes for 

 elections and decisions, the problems involved 

 deserve more compleite investigation. 



A probable error can be found for the posi- 

 tions of the individuals by a method that was 

 apparently first used by the present writer. 

 When eighty of those voting are divided into 

 ten groups of eight each, we have the separate 

 votes of each of these groups and from their 

 variation the probable error of the average vote 

 can be calculated. Thus the psychologist No. L, 

 in the ten groups of eight votes each received, 

 respectively, 4, 4, 2, 1, 4, 7, 4, 3, 2, and 2 votes. 

 The probable error is 0.363, and for the group 

 of eighty votes it is 3.63. The position on .the 

 curve assigned by each of the small groups can 

 also be found and a probable error calculated 

 from these ten positions. 



The pix)bable errors based on 80 votes (as a 

 rule for each fifth individual) are indicated by 

 the broken veritical line on the curve and when 

 referred to the order of merit by the broken 

 horizontal lines. The probable errors of the 

 votes of the five psychologists last in the fifty 

 are, respectively, 3.4, 2.8, 2.8, 3.1 and 3.6, an 

 average of 3.14. The curve from No. X to 

 No. C is nearly a straight line, the vote de- 

 creasing from 79' for No. X to 3 for No. C. 

 Consequently the probable error of the vote 

 when referred 'to the order is increased by 



