412 



BELL SYSTEM TECHNICAL JOURNAL 



All the e's and 6's implied by the words ^'small" and ''about" in these 

 statements approach zero as we allow T to increase and Sq to approach the 

 maximizing source. 



The situation is summarized in Fig. 10 where the input sequences are 

 points on the left and output sequences points on the right. The fan of 

 cross lines represents the range of possible causes for a typical output. 



Now suppose we have another source producing information at rate R 

 with R < C. In the period T this source will have 2^" high probability 

 outputs. We wish to associate these with a selection of the possible channe 



HIGH PROBABILITY^ 

 MESSAGES 



pHy(x)T 



"reasonable causes • 

 for each e 



2H(y)T 



HIGH PROBABILITY 

 RECEIVED signals 



• reasonable effects • 

 from each m 



Fig. 10 — Schematic representation of the relations between inputs and outputs in a 

 channel. 



inputs in such a way as to get a small frequency of errors. We will set up 

 this association in all possible ways (using, however, only the high proba- 

 bility group of inputs as determined by the source 5o) and average the fre- 

 quency of errors for this large class of possible coding systems. This is the 

 same as calculating the frequency of errors for a random association of the 

 messages and channel inputs of duration T. Suppose a particular output 

 yi is observed. What is the probability of more than one message in the set 

 of possible causes of yi? There are 2^'^ messages distributed at random in 

 2'^"^''^ points. The probability of a particular point being a message is 

 thus 



2T{R-H{x)) 



