The Bell System Technical Journal 



Vol. XXVII July, 194S No. 3 



A Mathematical Theory of Communication 



By C. E. SHANNON 



Introduction 



THE recent development of various methods of modulation such as PCM 

 and PPM which exchange bandwidth for signal-to-noise ratio has in- 

 tensified the interest in a general theory of communication. A basis for 

 such a theory is contained in the important papers of Nyquist^ and Hartley^ 

 on this subject. In the present paper we will extend the theory to include a 

 number of new factors, in particular the effect of noise in the channel, and 

 the savings possible due to the statistical structure of the original message 

 and due to the nature of the final destination of the information. 



The fundamental problem of communication is that of reproducing at 

 one point either exactly or approximately a message selected at another 

 point. Frequently the messages have meaning; that is they refer to or are 

 correlated according to some system with certain physical or conceptual 

 entities. These semantic aspects of communication are irrelevant to the 

 engineering problem. The significant aspect is that the actual message is 

 one selected from a set of possible messages. The system must be designed 

 to operate for each possible selection, not just the one which will actually 

 be chosen since this is unknown at the time of design. 



If the number of messages in the set is finite then this number or any 

 monotonic function of this number can be regarded as a measure of the in- 

 formation produced when one message is chosen from the set, all choices 

 being equally likely. As was pointed out by Hartley the most natural 

 choice is the logarithmic function. Although this definition must be gen- 

 eralized considerably when we consider the influence of the statistics of the 

 message and when we have a continuous range of messages, we will in all 

 cases use an essentially logarithmic measure. 



The logarithmic measure is more convenient for various reasons: 



1. It is practically more useful. Parameters of engineering importance 



1 Nyquist, H., "Certain Factors Affecting Telegraph Speed," Bell System TechnicdJour- 

 nal, April 1924, p. 324; "Certain Topics in Telegraph Transmission Theory," ^. /. E. E. 

 Ttans., V. 47, April 1928, p. 617. 



2 Hartley, R. V. L., "Transmission of Information," Bdl System TecttnicalJournal, July 

 1928, p. 535. 



379 



