Information Source is an object that produces a symbol or an event, where the outcome is random and is subjected to some probability distribution. In Communication System#, it is a device that produces messages in either analogue or digital form. In addition, analogue sources could be transformed into discrete form# by sampling and quantisation techniques.
It can be classified as either:
- A source with memory (current symbol depends on the previous symbols, or
- Memoryless Source#
When said information source (\(S\)) is a \(n\)-oder extension of itself, this means that it is \(S^n\), and it will produce Block Code# of size \(n\). The total number of block codes produced will be \(K^n\) where \(K\) is the number of individual symbols. Getting the probabilities for each block is as easy as multiplies each individual symbol’s probability. See details on Entropy# on how to calculate its uncertainty.