400 BELL SYSTEM TECHNICAL JOURNAL 



where: .r„ is the w"' input symbol, 



an is the state of the transducer when the n input symbol is introduced, 



yn is the output symbol (or sequence of output symbols) produced when 

 Xn is introduced if the state is a„. 



If the output symbols of one transducer can be identified with the input 

 symbols of a second, they can be connected in tandem and the result is also 

 a transducer. If there exists a second transducer w^hich operates on the out- 

 put of the first and recovers the original input, the first transducer will be 

 called non-singular and the second will be called its inverse. 

 Theorem 7: The output of a finite state transducer driven by a finite state 

 statistical source is a finite state statistical source, with entropy (per unit 

 time) less than or equal to that of the input. If the transducer is non- 

 singular they are equal. 



Let a represent the state of the source, which produces a sequence of 

 symbols Xi ; and let /3 be the state of the transducer, which produces, in its 

 output, blocks of symbols yj . The combined system can be represented 

 by the "product state space" of pairs (a, j8). Two points in the space, 

 (ai , j8i) and {ai^^)^ are connected by a line if (x\ can produce an x which 

 changes jSi to ^i , and this line is given the probability of that x in this case. 

 The line is labeled with the block of y^ symbols produced by the transducer. 

 The entropy of the output can be calculated as the weighted sum over the 

 states. If we sum first on ^ each resulting term is less than or equal to the 

 corresponding term for a, hence the entropy is not increased. If the trans- 

 ducer is non-singular let its output be connected to the inverse transducer. 

 If Ex , El and Ez are the output entropies of the source, the first and 

 second transducers respectively, then Ei > E2 ^ E^ = Ei and therefore 

 El = E2 . 



Suppose we have a system of constraints on possible sequences of the type 

 which can be represented by a linear graph as in Fig. 2. If probabilities 

 p^ij were assigned to the various lines connecting state i to state 7 this would 

 become a source. There is one particular assignment which maximizes the 

 resulting entropy (see Appendix IV). 



Theorem 8: Let the system of constraints considered as a channel have a 

 capacity C. If we assign 



where t^j is the duration of the i"* symbol leading from state i to state / 

 and the Bi satisfy 



then E is maximized and equal to C. 



