638 BELL SYSTEM TECHNICAL JOURNAL 



may assume an indeterminate form co — oo in some cases. This occurs, for 

 example, if x is limited to a surface of fewer dimensions than n in its n dimen- 

 sional approximation. 



If the logarithmic base used in computing F(x) and Hy{x) is two then C 

 is the maximum number of binary digits that can be sent per second over the 

 channel with arbitrarily small equivocation, just as in the discrete case. 

 This can be seen physically by dividing the space of signals into a large num- 

 ber of small cells, sufficiently small so that the probability density Px{y) 

 of signal x being perturbed to point y is substantially constant over a cell 

 (either of xox y). If the cells are considered as distinct points the situation 

 is essentially the same as a discrete channel and the proofs used there will 

 apply. But it is clear physically that this quantizing of the volume into 

 individual points cannot in any practical situation alter the final answer 

 significantly, provided the regions are sufficiently small. Thus the capacity 

 will be the limit of the capacities for the discrete subdivisions and this is 

 just the continuous capacity defined above. 



On the mathematical side it can be shown first (see Appendix 7) that if u 

 is the message, x is the signal, y is the received signal (perturbed by noise) 

 and V the recovered message then f 



H{x) - Hy{x) > H(u) - H,(u) 



regardless of what operations are performed on u to obtain x or on y to obtain 

 V. Thus no matter how we encode the binary digits to obtain the signal, or 

 how we decode the received signal to recover the message, the discrete rate 

 for the binary digits does not exceed the channel capacity we have defined. 

 On the other hand, it is possible under very general conditions to find a 

 coding system for transmitting binary digits at the rate C with as small an 

 equivocation or frequency of errors as desired. This is true, for example, if, 

 when we take a finite dimensional approximating space for the signal func- 

 tions, P{x, y) is continuous in both x and y except at a set of points of prob- 

 ability zero. 



An important special case occurs when the noise is added to the signal 

 and is independent of it (in the probability sense). Then Px{y) is a function 

 only of the difference n = {y — x), 



Pziy) = Qiy - oc) 



and we can assign a definite entropy to the noise (independent of the sta- 

 tistics of the signal), namely the entropy of the distribution Q(n). This 

 entropy will be denoted by H{n). 



Theorem 16: If the signal and noise are independent and the received 

 signal is the sum of the transmitted signal and the noise then the rate of 



