MATHEMATICAL THEORY OF COMMUNICATION 649 



tacitly assumed an evaluation based on the frequency of errors. The 

 function p{x, y) is then defined as the number of symbols in the sequence 

 y differing from the corresponding symbols in x divided by the total num- 

 ber of symbols in x. 



27. The Rate for a Source Relative to a Fidelity Evaluation 



We are now in a position to define a rate of generating information for a 

 continuous source. We are given P{x) for the source and an evaluation v 

 determined by a distance function p(x, y) which will be assumed continuous 

 in both X and y. With a particular system P(x, y) the quality is measured by 



"^ = jj p(^' y) ^(^' 3') ^^ ^y 



Furthermore the rate of flow of binary digits corresponding to P{x, y) is 



We define the rate Ri of generating information for a given quality vi of 

 reproduction to be the minimum of R when we keep v fixed at vi and vary 

 PxW. That is: 



subject to the constraint: 



vi = jj P(x, y)p{x, y) dx dy. 



This means that we consider, in effect, all the communication systems that 

 might be used and that transmit with the required fidelity. The rate of 

 transmission in bits per second is calculated for each one and we choose that 

 having the least rate. This latter rate is the rate we assign the source for 

 the fidelity in question. 



The justification of this definition lies in the following result: 



Theorem 21: If a source has a rate Ri for a valuation vi it is possible to 

 encode the output of the source and transmit it over a channel of capacity C 

 with fidelity as near vi as desired provided Ri < C. This is not possible 

 if Ri > C. 



The last statement in the theorem follows immediately from the definition 

 of Ri and previous results. If it were not true we could transmit more than 

 C bits per second over a channel of capacity C. The first part of the theorem 

 is proved by a method analogous to that used for Theorem 11. We may, in 

 the first place, divide the (x, y) space into a large number of small cells and 



