Basic Information Theory

Communication system performance is limited by • Available signal power • Background noise • Bandwidth limits. Can we postulate an ideal system based on physical principles, against which we can assess actual systems? The role of a communication system is to convey, from transmitter to receiver, a sequence of messages selected from a finite number of possible messages. Within a specified time interval, one of these messages is transmitted, during the next another (maybe the same one), and so on. • The messages are predetermined and known by the receiver • The message selected for transmission during a particular interval is not known by the receiver • The receiver knows the (different) probabilities for the selection of each message by the transmitter. The job of the receiver is not to answer the question “What was the message?”, but rather “Which one?”.