540 THE BELL SYSTEM TECHNICAL JOURNAL, MARCH 1957 



It is assumed that if the difference d between the best and second best 

 populations is small enough, then the error involved in wrongly select- 

 ing the second best process as the best one is an error of little or no con- 

 sequence. The experimenter is therefore asked to specify two quantities 

 which will determine the number n of observations he is required to 

 take from each process. 



Specification: He specifies the smallest value d* 

 (0 < cZ* ^ 1) of rf for which it would be economically 

 desirable to make the correct selection. He also specifies (4) 



a probability P* (0 ^ P* < 1) of making a correct 

 selection that he would like to guarantee whenever the 

 true difference d ^ d*. 



Letting Pes = Pes (?>[!] , • • • , Vw) denote the probability of a correct 

 selection we can now rewrite the specification that the experimenter 

 wants to satisfy in the simple form 



Pes ^ P* for d ^ d* (5) 



[The word "specification" will be used below to denote the specified 

 pair of constants (d*, P*) as well as the condition (5); it will be clear 

 from the text which is meant.] Since the final selection is to be made on 

 the basis of the observed frequency of success, the essential problem is 

 to find the number n of observations required per process to satisfy the 

 specification (5). 



The possibility that d may be less than d* is not being overlooked. 

 The region d < d* is being regarded as a zone of indifference in the 

 sense that ii d < d*, then we do not care which process is selected as best 

 so long as its p- value is within d* of the highest p- value pn] . For values 

 of P* S i/k no tables are needed since a probability of 1/k can be at- 

 tained by chance alone. 



Some comments on the above approach and on a possible modifica- 

 tion have been placed in Appendix I in order to preserve the conti- 

 nuity of the paper. 



CONFIDENCE STATEMENT 



After the experiment is completed and the selection of a best process 

 is made, the experimenter can make a confidence statement with confi- 

 dence level P*. Let ps denote the true p-"\'alue of the selected population 

 and let pa denote the maximum true /}-\'alue over all unselected popula- 



