646 BELL SYSTEM TECHNICAL JOURNAL 



PART V: THE RATE FOR A CONTINUOUS SOURCE 

 26. Fidelity Evaluation Functions 



In the case of a discrete source of information we were able to determine a 

 definite rate of generating information, namely the entropy of the under- 

 lying stochastic process. With a continuous source the situation is con- 

 siderably more involved. In the first place a continuously variable quantity 

 can assume an infinite number of values and requires, therefore, an infinite 

 number of binary digits for exact specification. This means that to transmit 

 the output of a continuous source with exact recovery at the receiving point 

 requires, in general, a channel of infinite capacity (in bits per second). 

 Since, ordinarily, channels have a certain amount of noise, and therefore a 

 finite capacity, exact transmission is impossible. 



This, however, evades the real issue. Practically, we are not interested 

 in exact transmission when we have a continuous source, but only in trans- 

 mission to within a certain tolerance. The question is, can we assign a 

 definite rate to a continuous source when we require only a certain fidelity 

 of recovery, measured in a suitable way. Of course, as the fidelity require- 

 ments are increased the rate will increase. It will be shown that we can, in 

 very general cases, define such a rate, having the property that it is possible, 

 by properly encoding the information, to transmit it over a channel whose 

 capacity is equal to the rate in question, and satisfy the fidelity requirements. 

 A channel of smaller capacity is insufficient. 



It is first necessary to give a general mathematical formulation of the idea 

 of fidelity of transmission. Consider the set of messages of a long duration, 

 say T seconds. The source is described by giving the probability density, 

 in the associated space, that the source will select the message in question 

 P{x). A given communication system is described (from the external point 

 of view) by giving the conditional probability Pxiy) that if message x is 

 produced by the source the recovered message at the receiving point will 

 be y. The system as a whole (including source and transmission system) 

 is described by the probability function P{x, y) of having message x and 

 final output y. If this function is known, the complete characteristics of 

 the system from the point of view of fidelity are known. Any evaluation 

 of fidelity must correspond mathematically to an operation applied to 

 P{x, y). This operation must at least have the properties of a simple order- 

 ing of systems; i.e. it must be possible to say of two systems represented by 

 Pi{x, y) and P-i.{x, y) that, according to our fidelity criterion, either (1) the 

 first has higher fidelity, (2) the second has higher fidelity, or (3) they have 



