Theoretical Limitations on the Rate of Transmission of Information

A review of early work on the theory of the transmission of information is followed by a critical survey of this work and a refutation of the point that, in the absence of noise, there is a finite limit to the rate at which information may be transmitted over a finite frequency band. A simple theory is then developed which includes, in a first-order way, the effects of noise. This theory shows that information may be transmitted over a given circuit according to the relation H ≤ 2BT log (1 + C/ N), where H is the quantity of information, B the transmission link bandwidth, T the time of transmission, and C/N the carrier-to-noise ratio. Certain special cases are considered, and it is shown that there are two distinctly different types of modulation systems, one trading bandwidth linearly for signal-to-noise ratio, the other trading bandwidth logarithmically for signal-to-noise ratio. The theory developed is applied to show some of the inefficiencies of present communication systems. The advantages to be gained by the removal of internal message correlations and analysis of the actual information content of a message are pointed out. The discussion is applied to such communication systems as radar relays, telemeters, voice communication systems, servomechanisms, and computers.