9722

Fisher Discriminant Analysis

The 30 round points are data. The 15 red points were generated from a normal distribution with mean , the 15 blue ones with mean , and in both cases the covariance matrix was the identity matrix. The problem is to classify or predict the color using the inputs and .
Fisher linear discriminant analysis determines a canonical direction for which the data is most separated when projected on a line in this direction. The solid gray line shows the canonical direction.
The squares are projected points on a line inclined at the angle with respect to the origin. When is adjusted so the projected points are aligned with the gray line, the points are maximally separated in the sense that the ratio of between-classes variances to within-classes variance is maximized.
A point is predicted as red or blue according to whether its projection on the canonical direction lies closest to the projected mean of the red or blue data points.

SNAPSHOTS

  • [Snapshot]
  • [Snapshot]
  • [Snapshot]
  • [Snapshot]

DETAILS

The canonical direction is given by
,
where and are the between- and within-classes covariance matrices. Hastie, Tibshirani and Friedman (2009, §4.3.3) [3] show that is given by the largest eigenvalue of .
The more general case where the number of inputs is greater than 2 is also considered in [3], but the basic principle of finding the canonical direction is the same. In our illustrative problem we have inputs as well as classes. In general, there are orthogonal canonical directions with the first canonical direction as defined above. Sometimes, as in [2], it is sufficient just to use just the first canonical component. For extensions, see [3].
[2] R. A. Fisher, "The Use of Multiple Measurements in Taxonomic Problems," Annals of Eugenics, 7, 1936 pp. 179–188.
[3] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., New York: Springer, 2009.
    • Share:

Embed Interactive Demonstration New!

Just copy and paste this snippet of JavaScript code into your website or blog to put the live Demonstration on your site. More details »

Files require Wolfram CDF Player or Mathematica.









 
RELATED RESOURCES
Mathematica »
The #1 tool for creating Demonstrations
and anything technical.
Wolfram|Alpha »
Explore anything with the first
computational knowledge engine.
MathWorld »
The web's most extensive
mathematics resource.
Course Assistant Apps »
An app for every course—
right in the palm of your hand.
Wolfram Blog »
Read our views on math,
science, and technology.
Computable Document Format »
The format that makes Demonstrations
(and any information) easy to share and
interact with.
STEM Initiative »
Programs & resources for
educators, schools & students.
Computerbasedmath.org »
Join the initiative for modernizing
math education.
Step-by-step Solutions »
Walk through homework problems one step at a time, with hints to help along the way.
Wolfram Problem Generator »
Unlimited random practice problems and answers with built-in Step-by-step solutions. Practice online or make a printable study sheet.
Wolfram Language »
Knowledge-based programming for everyone.
Powered by Wolfram Mathematica © 2014 Wolfram Demonstrations Project & Contributors  |  Terms of Use  |  Privacy Policy  |  RSS Give us your feedback
Note: To run this Demonstration you need Mathematica 7+ or the free Mathematica Player 7EX
Download or upgrade to Mathematica Player 7EX
I already have Mathematica Player or Mathematica 7+