64 THE BELL SYSTEM TECHNICAL JOURNAL, JANUARY 1951 



The term in brackets has the form —f{x — 1) + 2f{x) — f{x +1) where 

 f{x) = X log X. Now/(x) is a function which is concave upward for positive x, 

 since/" (x) = l/o; > 0. The bracketed term is twice the difference between the 

 ordinate of the curve Sit x = i and the ordinate of the midpoint of the chord 

 joining i — 1 and i + 1, and consequently is negative. Since A^ also is nega- 

 tive, the change in U brought about by the flow is positive. An even simpler 

 calculation shows that this is also true for a flow from qi to ^2 or from ^26 to 

 ^27 (where only two terms of the sum are affected). It follows that the lower 

 bound based on the iV-gram prediction frequencies q^ is greater than or 

 equal to that calculated from the 7\^ + 1 gram frequencies q^'^^ . 



6. Experimental Bounds for English 



Working from the data of Table I, the upper and lower bounds were calcu- 

 lated from relations (17). The data were first smoothed somewhat to over- 

 come the worst sampling fluctuations. The low numbers in this table are 

 the least reliable and these were averaged together in groups. Thus, in 

 column 4, the 47, 18 and 14 were not changed but the remaining group 

 totaling 21 was divided uniformly over the rows from 4 to 20. The upper and 

 lower bounds given by (17) were then calculated for each column giving the 

 following results: 



Column 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 100 



Upper 4.03 3.42 3.0 2.6 2.7 2.2 2.8 1.8 1.9 2.1 2.2 2.3 2. 1 1.7 2.1 1.3 



Lower 3.192.502.11.71.71.31.81.01.01.01.31.31.2 .91.2 .6 



It is evident that there is still considerable sampling error in these figures 

 due to identifying the observed sample frequencies with the prediction 

 probabilities. It must also be remembered that the lower bound was proved 

 only for the ideal predictor, while the frequencies used here are from human 

 prediction. Some rough calculations, however, indicate that the discrepancy 

 between the actual F^ and the lower bound with ideal prediction (due to 

 the failure to have rectangular distributions of conditional probability) 

 more than compensates for the failure of human subjects to predict in the 

 ideal manner. Thus we feel reasonably confident of both bounds apart from 

 sampling errors. The values given above are plotted against N in Fig. 4. 



Acknowledgment 



The writer is indebted to Mrs. Mary E. Shannon and to Dr. B. M. Oliver 

 for help with the experimental work and for a number of suggestions and 

 criticisms concerning the theoretical aspects of this paper. 



