Benford's Law and Data Spread

Requires a Wolfram Notebook System

Interact on desktop, mobile and cloud with the free Wolfram CDF Player or other Wolfram Language products.

Requires a Wolfram Notebook System

Edit on desktop, mobile and cloud with any Wolfram Language product.

Benford's law is the observation that for many datasets, the distribution of their first significant digit follows a nonuniform law given by:

[more]

Probability(leading digit = ) = .

Thus the probability that the leading digit is 1 is 30%, while for the digit 7 to lead, the probability is merely 6%. An underlying reason for this is that data that spans many orders of magnitude has the errors within each order cancel out (see Details section). Thus datasets with large logarithmic spread will naturally follow the law, while datasets with small spread will not.

This Demonstration shows any of the scatter plots of 130 datasets derived from the data on countries in Mathematica; the points in a scatter plot are of the form (logarithmic spread, Benford deviation). Here spread is computed by taking base-10 logarithms and eliminating extreme outliers; the Benford deviation is the norm of the vector difference of the observed frequencies and the Benford predictions, normalized to lie between 0 and 1. Below the scatter plot are plots of the raw distribution, the agreement of the digit probabilities with Benford's law, and the distribution of the base-10 logarithms of the data. Note that the scatter plot supports the explanation remarkably well: all properties with large spread have small Benford deviation, and all properties with small spread have large Benford deviation.

[less]

Contributed by: Stan Wagon (Macalester College) (March 2009)
Open content licensed under CC BY-NC-SA


Snapshots


Details

The connection between Benford's law and the spread of the data is lucidly described in [1] (see also Chapter 21 of [3] and Chapter 1 of [3]). The idea is that when viewing a distribution in a base-10 log scale, the proportion of the axis corresponding to numbers beginning with 1 is . Hence, so long as there are several orders of magnitude, the error between this proportion and the proportion of the area lying above these numbers should be small.

[1] R. M. Fewster, "A Simple Explanation of Benford's Law," The American Statistician, 63(1), 2009 pp. 26-32.

[2] Benford's Law: Theory and Applications, edited by Steven J. Miller, Princeton: Princeton Univ. Pr., 2015.

[3] S. Wagon, Mathematica in Action, 3rd ed., New York: Springer, 2010.



Feedback (field required)
Email (field required) Name
Occupation Organization
Note: Your message & contact information may be shared with the author of any specific Demonstration for which you give feedback.
Send