Least Squares

When a matrix A is square with full rank, there is a vector that satisfies the equation for any . However, when A is not square or does not have full rank, such an may not exist, because b does not lie in the range of A. In this case, called the least squares problem, we seek the vector x that minimizes the length (or norm) of the residual vector . The four vectors , , , and are color coded and the plane is the range of the matrix . The plane shown is the set of all possible vectors .