384 BELL SYSTEM TECHNICAL JOURNAL 



a space \vas the last symbol transmitted. If so then only a dot or a dash 

 can be sent next and the state always changes. If not, any symbol can be 

 transmitted and the state changes if a space is sent, otherwise it remains 

 the same. The conditions can be indicated in a linear graph as shown in 

 Fig. 2. The junction points correspond to the states and the lines indicate 

 the symbols possible in a state and the resulting state. In Appendix I it is 

 shown that if the conditions on allowed sequences can be described in this 

 form C will exist and can be calculated in accordance with the following 

 result: 



Theorem 1: Let hi) be the duration of the s^^ symbol which is allowable in 

 state i and leads to state j. Then the channel capacity C is equal to log 

 W where W is the largest real root of the determinant equation : 



where ba = \ iii = j and is zero otherwise. 



DASH 



DOT 



DASH 



WORD SPACE 

 Fig. 2 — Graphical representation of the constraints on telegraph symbols. 



For example, in the telegraph case (Fig. 2) the determinant is: 



-1 {w-^ + w') 



{W~^ + W') (IF"' + W~' - 1) 

 On expansion this leads to the equation given above for this case. 



2. The Discrete Source of Information 



We have seen that under very general conditions the logarithm of the 

 number of possible signals in a discrete channel increases linearly with time. 

 The capacity to transmit information can be specified by giving this rate of 

 increase, the number of bits per second required to specify the particular 

 signal used. 



We now consider the information source. How is an information source 

 to be described mathematically, and how much information in bits per sec- 

 ond is produced in a given source? The main point at issue is the effect of 

 statistical knowledge about the source in reducing the required capacity 



= 



