728 THE BELL SYSTEM TECHNICAL JOURNAL, JULY 1952 



less impractical when the signal to noise ratios are already high What 

 we should really try to do is make our descriptions of our messages more 

 efficient so that less channel capacity is required in the first place. The 

 saving can then be taken either in bandwidth or in signal-to-noise ratio, 

 whichever fits the requirements of our channels best. 



MESSAGES 



Messages can either be continuous waves like speech, music, or tele- 

 vision; or they can consist of a succession of discrete characters each with 

 a finite set of possible values, such as English text. Because a finite band- 

 width and a small added noise are both permissible, continuous signals 

 can be converted to discrete signals by the processes of sampling and 

 quantizing . This permits us to talk about them as equivalent from the 

 communication engineering viewpoint. Since many of the principles 

 which follow are easier to think of with discrete messages and since 

 quantization of the channel is assumed for reasons already stated, we 

 shall think of our messages as always being available in discrete form. 



Let S = the symbol (or sample) rate of the message 



S 

 Wo = K = the original bandwith of the continuous message 



^ = number of quantizing levels. 



Then if all the message samples were independent and if all quantizing 

 levels were equally likely, the information per sample would be 



Ho = log2 ( bits (3) 



the information rate would be 



Ho = S logo ( bits/sec (4) 



and the message would use the full capacitj^ of a channel with / quan- 

 tizing levels, and bandwidth S/2. Or by remapping k message samples 



/log A 

 (with the i possible levels) into I .-^—r I k samples, a channel mth b levels 



yog b/ 



/log A 

 and bandwidth W = S/2 ( ,-^— I could be loaded to full capacity. 



yog b/ 



However, it is not true that the successive samples of typical messages 



are independent, nor is it true that the various sample amplitudes are 



in general equiprobable. If these things were tiTie, speech and music 



would sound like white noise, pictiu'es would look like the snowstorm 



