Communication System

The purpose of a Communication System is to transmit information from one point to another with high efficiency and reliability that is acceptable to user. With the use of Source Coding Theorem#, the communication channel could be more efficient since the redundancy are reduced from the message signals. With the use of Channel Coding Theorem#, the reliability of the transmission will be improved even though it also introduces systematic redundancy.

Unfortunately, in practice, there will always be noise and other transmission impairment that influence the source symbol in the channel, which retain some uncertainties even after observing the output. We described them with Conditional Entropy# and Mutual Information#.

Links to this page

TIT3131 Chapter 3: Linear Block Coding

Communication System
TIT3131 Chapter 1: Information Sources and Sources Coding

Communication System
Pipe

Pipe provides a one-way flow of data, where the output of a process is directed into the input of another, which can be created using pipe() #Unix System Call. Pass in a file descriptor array (of type int) with a size of 2, the system call will return 2 file descriptors for use: fd[0] for reading and fd[1] for writing. A typical use case of Pipe is to establish a communication channel# between two related processes. The maximum amount of data allowed in the pipe (OS treats it like a Queue#) is usually 5120 bytes. In #unix, Pipe is implemented using #Unix Domain Sockets.
Information Theory

Information Theory is a quantitative measure of the information# contained in message signals and allows us to determine the capacity of the communication system to transmit information# from source to destination. It deals with the mathematical modelling and analysis of a communication system#. The laws of probability are applied to Information Theory. It covers Source Coding Theorem# and Channel Coding Theorem#.
Information Source

Information Source is an object that produces a symbol or an event, where the outcome is random and is subjected to some probability distribution. In Communication System#, it is a device that produces messages in either analogue or digital form. In addition, analogue sources could be transformed into discrete form# by sampling and quantisation techniques.
Discrete Memoryless Channel (DMC)

DMC is a #communication channel having two alphabets (source): input \(X\) and output \(Y\) which has the following characteristics:
Conditional Entropy

Conditional Entropy represents the amount of uncertainty remaining about the #channel input after the channel output has been observed (which is different from Entropy which measure the information in the source). It is defined mathematically by:
Channel Coding Theorem

Channel Coding Theorem, or Shannon’s second theorem is a safe or maximum limit of transmission rate over a noisy channel without the loss of Information#. In fact, the probability of error should be \(10^{-6}\) or even lower so that transmission of the symbols is feasible. Channel Coding is to increase the resistance of a digital Communication System# to channel noise in order to achieve high performance level. It is by increasing the redundancy into the code.

#networking