Karhunen-Loeve Directions and Regression

Students sometimes ask about the difference between the regression line and the Karhunen–Loeve direction. The obvious answer a professor might give is that they are different animals! The object of this Demonstration is to give a more interesting answer.
The case of variables, and , is shown. It is assumed that and are bivariate normal with means zero, variances one, and covariance . For , the expected values for the regression lines on and on are shown together with the theoretical Karhunen–Loeve directions. For the estimated regressions and directions are shown along with a scatter plot of the data.

SNAPSHOTS

  • [Snapshot]
  • [Snapshot]
  • [Snapshot]

DETAILS

Consider the multivariate random variable with mean and covariance matrix . Then the Karhunen–Loeve directions are determined by columns of in the eigendecomposition , while the regression of on is
,
where is the matrix with the first row and column removed. In the present example, this simplifies to and similarly for the regression of on .
The related question based on data may be asked. In this case we just replace expectations by their sample estimates.
    • Share:

Embed Interactive Demonstration New!

Just copy and paste this snippet of JavaScript code into your website or blog to put the live Demonstration on your site. More details »

Files require Wolfram CDF Player or Mathematica.