Efficient Coding 



By B. M. OLIVER 



(Manuscript received May 14, 1952) 



This paper reviews hrieflij a few of the simpler aspects of communication 

 theory, especially those parts which relate to the information rate of and 

 channel capacity required for sampled, quantized messages. Two methods 

 are then discussed, whereby such messages can be converted to a "reduced" 

 form in which the successive samples are more nearly independent and for 

 which the simple amplitude distribution is more peaked than in the original 

 message. This reduced signal can then be encoded into binary digits with 

 good efficiency using a Shannon-Fano code on a symbol-by-symbol {or 

 pair-by-pair) basis. The usual inefficiency which results from ignoring 

 the correlation between message segments is lessened because this correlation 

 is less in the reduced message. 



INTRODUCTION 



The term coding, as applied to electrical communication, has several 

 meanings. It means the representation of letters as sequences of dots 

 and dashes. It means the representation of signal sample amplitudes as 

 groups of pulses ha^dng two or more possible amplitudes as in pulse 

 code modulation. Lately, it has also come to be the generic term for 

 any process by which a message or message wave is converted into a 

 signal suitable for a given channel. In this usage single-sideband modula- 

 tion, frequency modulation and pulse code modulation are examples of 

 encoding procedures, while microphones, teletypewriters and television 

 cameras are examples of encoding devices. 



This is a nice concept, but it is useful to distinguish between two classes 

 of encoding processes and devices: those which make no use of the 

 statistical properties of the signal, and those which do. In the first class, 

 the encoding operation consists simply of a one-to-one conversion of 

 the message into a new physical variable, as a microphone converts sound 

 pressure into a proportional voltage or current, or of the one-to-one 

 remapping of the message into a new representation without regard to 

 probabilities, as by ordinary amplitude, frequency or pulse code modula- 



724 



