Variance-Bias Tradeoff

The plot shows the true density function , which is normal with mean 0 and variance 1, and its nonparametric estimate obtained using a kernel smoothing with parameter . The initial value for of 0.95 corresponds to the kernel smoother that minimizes the integrated mean-square error, IMSE. Graphically, IMSE is the area between the two curves. Smaller values correspond to less smoothing and larger to more smoothing. With less smoothing, the red curve wobbles more around the true value, but there is less systematic bias.
Although this Demonstration only deals explicitly with an example nonparametric density estimation, the situation is a paradigm for all empirical statistical model building.


  • [Snapshot]
  • [Snapshot]
  • [Snapshot]


The variance-bias tradeoff is most simply explained mathematically in terms of estimating a single parameter with an estimator . Then the mean-square error of estimation for provides an estimate of the accuracy of the estimator and is defined by ,
where denotes mathematical expectation. The bias is defined by and the variance is ; hence . Thus there are two components to the error of estimation—one due to bias and the other variance.
This paradigm is very general and includes all statistical modelling problems involving smoothing or parameter estimation. For a more general discussion of this aspect, see §2.9 of T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., New York: Springer, 2009.
    • Share:

Embed Interactive Demonstration New!

Just copy and paste this snippet of JavaScript code into your website or blog to put the live Demonstration on your site. More details »

Files require Wolfram CDF Player or Mathematica.

Mathematica »
The #1 tool for creating Demonstrations
and anything technical.
Wolfram|Alpha »
Explore anything with the first
computational knowledge engine.
MathWorld »
The web's most extensive
mathematics resource.
Course Assistant Apps »
An app for every course—
right in the palm of your hand.
Wolfram Blog »
Read our views on math,
science, and technology.
Computable Document Format »
The format that makes Demonstrations
(and any information) easy to share and interact with.
STEM Initiative »
Programs & resources for
educators, schools & students.
Computerbasedmath.org »
Join the initiative for modernizing
math education.
Powered by Wolfram Mathematica © 2014 Wolfram Demonstrations Project & Contributors  |  Terms of Use  |  Privacy Policy  |  RSS Give us your feedback
Note: To run this Demonstration you need Mathematica 7+ or the free Mathematica Player 7EX
Download or upgrade to Mathematica Player 7EX
I already have Mathematica Player or Mathematica 7+