Nt1310 Unit 4 Case Study

.5 New procedure for messages and average amount of information: We have determined in the preceding section the amount of information conveyed by a single selection from M equally likely choices. In general, however, we have to deal with not one but long series of such selection, which we call messages. This is the case, for instance, in the transmission of written intelligence. Another example is provided by the communication system known as pulse-code modulation, in which audio waves are sampled at equal time intervals and then each sample is equal is quantized, that is approximated by the closest of a number M of amplitude levels. Let us consider, then, a message consisting

Implement Fano method. Given a set of messages with their probabilities, your program should output codes for each message.

6

Sometimes, instead of probabilities we know frequencies of messages – how often a message appears in a certain text. In this case, we can computer probabilities by assuming that they are proportional to corresponding frequencies.

In other words, the probability of a message is set to be the frequency of the message divided by the sum of frequencies of all messages.

However, in the Shannon Fano coding procedure we do not need to compute probabilities, we can use frequencies instead. A node can be either a leaf node or an internal node. Initially, all nodes are leaf nodes, which contain the symbol itself, the weight (frequency of appearance) of the symbol and optionally, a link to a parent node which makes it easy to read the code (in reverse) starting from a leaf node. Internal nodes contain a weight, links to two child nodes and an optional link to a parent node.

As a common convention, bit '0' represents following the left child and bit '1' represents following the right child. A finished tree has up to leaf nodes and internal nodes. A Huffman tree that omits unused symbols produces the most optimal code