MATHEMATICAL THEORY OF COMMUNICATION 383 



In the more general case with different lengths of symbols and constraints 

 on the allowed sequences, we make the following definition: 

 Definition: The capacity C of a discrete channel is given by 



r-»oo 1 



where N{T) is the number of allowed signals of duration T. 



It is easily seen that in the teletype case this reduces to the previous 

 result. It can be shown that the limit in question will exist as a finite num- 

 ber in most cases of interest. Suppose all sequences of the symbols 5i , • • • , 

 Sn are allowed and these symbols have durations h , ■ • - , tn . What is the 

 channel capacity? If N(t) represents the number of sequences of duration 

 / we have 



N{t) = N{t - h) + N{t- t2)+ -■' + N{t - tn) 



The total number is equal to the sum of the numbers of sequences ending in 

 Si, S2, • • ' , Sn and these are Nit — /i), N{t — /2), • • • , N{t — tn), respec- 

 tively. According to a well known result in finite differences, N(t) is then 

 asymptotic for large / to Xj where Xq is the largest real solution of the 

 characteristic equation: 



X~'' + X~'' + "■ + X"'" = 1 

 and therefore 



C = log Xo 



In case there are restrictions on allowed sequences we may still often ob- 

 tain a difference equation of this type and find C from the characteristic 

 equation. In the telegraphy case mentioned above 



Nit) = N{t - 2) + Nit - 4) -1- N{t - 5) + N{t - 7) -f N{t - 8) 



-f N{t - 10) 



as we see by counting sequences of symbols according to the last or next to 

 the last symbol occurring. Hence C is — log /io where /io is the positive 

 root of 1 = m' + m' + m' + m' + m' + /i''. Solving this we find C = 0.539. 

 A very general type of restriction which may be placed on allowed se- 

 quences is the following : We imagine a number of possible states fli , ^2 , • • • , 

 Gm . For each state only certain symbols from the set 5i , • • • , Sn can be 

 transmitted (different subsets for the different states). When one of these 

 has been transmitted the state changes to a new state depending both on 

 the old state and the particular symbol transmitted. The telegraph case is 

 a simple example of this. There are two states depending on whether or not 



