Dimensionality reduction algorithms differ in the constraints they impose on the new variables and how they aim to minimize the loss of information:
- Linear algorithms such as PCA and ICA constrain the new variables to be linear combinations of the original features; that is, hyperplanes in a lower-dimensional space. Whereas PCA requires the new features to be uncorrelated, ICA goes further and imposes statistical independence—the absence of both linear and non-linear relationships. The following screenshot illustrates how PCA projects three-dimensional features into a two-dimensional space:
- Non-linear algorithms ...
