Back to main page

Entropy Coding

After the data has been quantized into a finite set of values, it can be encoded using an Entropy Coder to give additional compression. By entropy, we mean the amount of information present in the data, and an entropy coder encodes the given set of symbols with the minimum number of bits required to represent them.

Two of the most popular entropy coding schemes are Huffman coding and Arithmetic coding. 

A good introduction to Huffman coding can be found in 

• "A Closer Look at Huffman Encoding" at http://www.rasip.fer.hr/research/compress/algorithms/fund/huffman/

A good introduction to Arithmetic coding can be found in 

• "Arithmetic Coding+Statistical Modelling=Data Compression" at http://dogma.net/markn/articles/arith/part1.htm
• H.Witten, M.Neal and G.Cleary, "Arithmetic Coding For data Compression",
  Communications of the ACM, Vol. 30, No. 6 (June 1987), Pages 520-540.

Bit allocation

Source code