11453

Parametric Density Estimation Using Polynomials and Fourier Series

Imagine you obtain samples of two types, such as the red and green points in the graph shown here, and you would like to generalize by classifying some other points into one of these two classes. This is a basic problem in machine learning called supervised classification, with the class of each sampled point known. The standard approach is based on linear classifiers: using (hyper) planes for separation into two classes, analogous to its use in support vector machines or neural networks.
This Demonstration lets you play with direct nonlinear (two-class) classifiers in a 2D square: fitting a polynomial or Fourier series to the sample, using weights of opposite signs to distinguish the classes, followed by classification based on the sign of the difference of densities.
You can add points from the class of a chosen color by clicking its position. You can also generate some chosen number of random points. The "degree" controller lets you choose the degree of the polynomial. The number of coefficients used is shown below it. The "method" controller lets you switch between fitting a polynomial or a Fourier series. The "plot" controller lets you choose between RegionPlot and DensityPlot.

THINGS TO TRY

SNAPSHOTS

  • [Snapshot]
  • [Snapshot]
  • [Snapshot]

DETAILS

The method used for fitting density is described in [1]. The formula was derived by smoothing the sample through convolution with a kernel, then fitting a polynomial or Fourier series to the smoothed sample. It turns out that for mean-square fitting, we can perform the limit to a zero-width kernel, giving asymptotically optimal coefficients with a very simple formula.
Specifically, assuming orthonormal bases of functions (Legendre polynomials or sines and cosines here), the linear coefficient for a given function turns out to be just the average of the function over the sample:
,
where is the average of over the sample.
For the classification problem in this Demonstration, this density for green points was subtracted from the density for red points. Either the region of positive values (RegionPlot) or its density (DensityPlot) is then drawn.
Reference
[1] J. Duda, "Rapid Parametric Density Estimation." (Mar 14, 2017) arxiv.org/abs/1702.02144.
    • Share:

Embed Interactive Demonstration New!

Just copy and paste this snippet of JavaScript code into your website or blog to put the live Demonstration on your site. More details »

Files require Wolfram CDF Player or Mathematica.









 
RELATED RESOURCES
Mathematica »
The #1 tool for creating Demonstrations
and anything technical.
Wolfram|Alpha »
Explore anything with the first
computational knowledge engine.
MathWorld »
The web's most extensive
mathematics resource.
Course Assistant Apps »
An app for every course—
right in the palm of your hand.
Wolfram Blog »
Read our views on math,
science, and technology.
Computable Document Format »
The format that makes Demonstrations
(and any information) easy to share and
interact with.
STEM Initiative »
Programs & resources for
educators, schools & students.
Computerbasedmath.org »
Join the initiative for modernizing
math education.
Step-by-Step Solutions »
Walk through homework problems one step at a time, with hints to help along the way.
Wolfram Problem Generator »
Unlimited random practice problems and answers with built-in step-by-step solutions. Practice online or make a printable study sheet.
Wolfram Language »
Knowledge-based programming for everyone.
Powered by Wolfram Mathematica © 2017 Wolfram Demonstrations Project & Contributors  |  Terms of Use  |  Privacy Policy  |  RSS Give us your feedback
Note: To run this Demonstration you need Mathematica 7+ or the free Mathematica Player 7EX
Download or upgrade to Mathematica Player 7EX
I already have Mathematica Player or Mathematica 7+