EFFICIENT CODING 727 



use only a tiny fraction of this vocabulary. If all the letters of the written 

 alphabet were used with ef(ual prol)ablUty and if all combinations of 

 letters were allowed, then many words which are now long could l)e 

 made shorter, and written text would be less than one third as long as 

 English. Similarly, if we could arrange to let our circuits use their en- 

 tire vocabulary with equal probability, they could describe our messages 

 with much less time (or bandwidth) on the average. 



EXCHANGE OF BANDWIDTH AND SIGNAL TO NOISE RATIO 



It was the advent of wide band ¥M, and other modulation methods 

 which exchange bandwidth for signal-to-noise ratio, which revealed the 

 inadequacy of earlier concepts of information transmission and ulti- 

 mately led to the development of modern communication theory, or 

 information theory . 



One of tlie more familiar results of this theory is the expression for the 

 maximum capacity of a channel (listurl)ed by white noise: 



C =W log, (^1 + ^^ (2) 



in which C is the capacity in bits/sec, W is the bandwidth and P/N the 

 ratio of average signal power to average noise power. This capacity can 

 only be approached, never exceeded, and is only reached when the sig- 

 nal itself has the statistics of a white noise. The expression sets a limit 

 for practical endeavor, and also gives the theoretical rate of exchange 

 between W and P/N. 



A practical quantized channel, operated so that the loss of informa- 

 tion due to incorrectly received levels is negligible requires about 20 

 db more peak signal power than the average signal power of the ideal 

 channel to attain the same capacity \ However, bandwidth and signal- 

 to-noise ratio are still exchanged on the same basis. For example, a satis- 

 factory tele\Tlsion picture could be sent over a channel with, say, 100 

 levels. This would require a (peak) signal to rms noise ratio of some 40 + 

 20 = 60 db. The bandwidth could be halved by a sort of reverse PCM: 

 by using one pulse- to represent two picture elements. But there are 10,000 

 combinations of two samples each of which can have any of 100 values. 

 Hence the new combination pulse would need 10,000 distinguishable 

 levels and this would require a signal to noise ratio of 80 + 20 = (2 X 

 40) + 20 = 100 db. 



It is evident that while bandwidth compression by non-statistical or 

 straight signal remapping means is not an impossibility, it is neverthe- 



