EFFICIENT CODING 725 



tion. In ordinary PCINI for example, the message samples are converted 

 into groups of on-or-otf pulses. The particular combination of pulses 

 in any group depends only upon the amplitude of the particular sample, 

 not upon any other property of the message, and the same time is allotted 

 to each group, regardless of the probability of that group or of the 

 amplitude it represents. Almost all the processes and devices used in 

 present day communication })el()ng to this first class. In the second class, 

 the probabilities of the message are taken into account so that short 

 representations are used for likely messages or likely subsequences, longer 

 representations for less likely ones. Morse code, for example, uses short 

 code groups for the common letters, longer code groups for the rare ones. 



Processes of the first class we may call non-statistical coding processes, 

 or simply modulation or remapping processes. The time of transmission 

 is the same for all messages of the same length, and all messages are 

 handled by the sj^stem with eciual facility (or difficulty). These processes 

 require no memory and have a small and constant delay They are 

 inefficient in their use of channel capacity. 



Processes of the second class we may call statistical encoding proc- 

 esses. These processes in general recjuire memory. The time of trans- 

 mission of messages of the same length may be different so that if 

 messages are to be accepted and delivered by the system at constant 

 rates, variable delays may be necessary at the sending and receiving 

 ends. They are more efficient in their use of channel capacity. It is with 

 this second type of process that this paper is concerned, although pro- 

 cesses of the first type may be used as component steps. Thus we con- 

 sider systems of the type shown in Fig. 1, with the accent on the word 

 "efficient". 



TRANSMISSION CIRCUITS AND THEIR VOCABULARIES 



C'ommvuiication circuits or channels can, of course, differ in many 

 respects. Either the peak signal power or the average signal power may 

 be limited. The transmission may be uniform over the band or vary 

 with fre(iuency; it may be constant or subject to selective fading. The 

 noise may be gaussian thermal or shot noise uniform across the band, or 

 peaked at some frefjuency; or it may be largely impulse noise or erratic 



SIGNAL= EFFICIENT DESCRIPTION OF MESSAGE 



Fig. 1 — Reversible statistical encoding. 



