Probabilities are used to characterize the likelihood of events or propositions. In some circumstances, predictions of probability carry a high degree of confidence. For example, an individual can confidently predict that a fair coin will produce “heads” in one flip with probability

. By way of contrast, there is more uncertainty associated with a weather prediction that states the probability of rain tomorrow as

. E. T. Jaynes developed the concept of the

distribution to deal with what he described as different states of external and internal knowledge. In the terminology of Jaynes, the probability of the proposition

is found by computing the mean of the

distribution, and the variance of the

distribution is a measure of the amount of confidence associated with the prediction of the mean. In situations where you have high states of internal knowledge, like the case of the coin, the variance of the

distribution is small. In fact, for the case of coin, the variance of the

distribution is 0.

The entropy

is a measure of the amount of disorder in a probability density function. The principle of maximum entropy can be used to find

distributions in circumstances where the only specified information is the mean of the distribution or the mean and variance of the distribution. The

distributions in this Demonstration are evaluated at the points

for

. If the probability density at these

points is denoted by

, then the mean

, variance

, and entropy

of the

distribution are respectively given by

If the mean

of the

distribution is specified, then the corresponding maximum entropy probability distribution

can be found using the technique of Lagrange multipliers [2]. This requires finding the maximum of the quantity

,

where the unknowns are the probabilities

and the Lagrange multipliers

and

. If the mean

and the variance

of the

distributions are both specified, then it is necessary to find the maximum value of the quantity

,

where

is an additional Lagrange multiplier.

[1] E. T. Jaynes,

*Probability Theory: The Logic of Science*, New York: Cambridge University Press, 2003.

[2] P. Gregory,

*Bayesian Logical Data Analysis for the Physical Sciences*, Cambridge: Cambridge University Press, 2005.