A Limit Theorem from Information Theory
Requires a Wolfram Notebook System
Interact on desktop, mobile and cloud with the free Wolfram Player or other Wolfram Language products.
How many distinct sequences can be created by rearranging binary symbols? If the binary symbols are, say, and , and if the fraction of 's is denoted , then the number of possible sequences is . When is zero or one there is only one possible sequence for any value of , but when , the number of possible sequences increases exponentially with . The logarithm of the number of possible sequences, expressed on a per symbol basis, is , and the limit can be interpreted as the average number of bits needed per symbol to describe a long binary sequence with symbol probabilities and . This Demonstration shows for with the limit .
Contributed by: John Custy (November 2008)
Open content licensed under CC BY-NC-SA
"A Limit Theorem from Information Theory"
Wolfram Demonstrations Project
Published: November 30 2008