Principal Component Analysis

Target

PCA tries to get a projection panel by minimizing the projection error (not error in linear regression)

How to implement

  • Calculate Sigma \Sigma by $\Sigma = \frac{1}{m}X^TX$
  • Get projection panel U (along with error S ) by singular value decomposition of \Sigma
  • Get new data set Z by Z = X * U
  • Recover original data (approximated) by X_{approx} = Z * U^T