MATHEMATICAL THEORY OF COMMUNICATION 647 



equal fidelity. This means that a criterion of fidelity can be represented by 

 a numerically valued function: 



v{P{x, y)) 



whose argument ranges over possible probability functions F{x, y). 



We will now show that under very general and reasonable assumptions 

 the function v{P{x, y)) can be written in a seemingly much more specialized 

 form, namely as an average of a function p{x, y) over the set of possible values 

 of X and y: 



)(P{x, y)) = j j P{x, y) p{x, y) dx dy 



To obtain this we need only assume (1) that the source and system are 

 ergodic so that a very long sample will be, with probability nearly 1, typical 

 of the ensemble, and (2) that the evaluation is ' 'reasonable" in the sense 

 that it is possible, by observing a typical input and output Xi and yi , to 

 form a tentative evaluation on the basis of these samples; and if these 

 samples are increased in duration the tentative evaluation will, with proba- 

 bility 1, approach the exact evaluation based on a full knowledge of P{x, y). 

 Let the tentative evaluation be p(x, y). Then the function p(x, y) ap- 

 proaches (as r -^ 00 ) a constant for almost all (x, y) which are in the high 

 probability region corresponding to the system: 



p(x, y) -^ v(P(x, y)) 

 and we may also write 



p{x, y) -^ j j P{x, y)pix, y) dx, dy 



smce 



/ / P{x, y) dx dy = 1 



This establishes the desired result. 



The function p{x, y) has the general nature of a "distance" between x 

 and y. It measures how bad it is (according to our fidelity criterion) to 

 receive y when x is transmitted. The general result given above can be 

 restated as follows: Any reasonable evaluation can be represented as an 

 average of a distance function over the set of messages and recovered mes- 

 sages X and y weighted according to the probability P(x, y) of getting the 

 pair in question, provided the duration T of the messages be taken suffi- 

 ciently large. 



®It is not a "metric" in the strict sense, however, since in general it does not satisfy 

 either p(x, y) = p{y, x) or p{x, y) + p{y, z) > p{x, z). 



