Włodzimierz Kasprzak. My Home

Adaptive PCA and PSA algorithms

ANN for Principal Component Analysis


The CRLS PCA method

We have developed a new fast learning algorithm for neural network based PCA, called CRLS (cascade recurrent least square) PCA

Consider our important paper published in "Cybernetics and Systems", 1996.

The main features of this learning algorithm are: sequential PC extraction with a Hebbian like learning rule, deflation of input signals, an RLS scheme for automatic adaptation of learning rate.

Thus CRLS PCA is a simple neural network learning method that outperforms other similar neural network methods in the speed-to-quality factor, and that properly extracts all components (not only the highest range but the intermediate range and minor components also). Although for experimental validity of the method an image compression-reconstruction cycle was usually used, an image compression application was definitely NOT our motivation, as there are already very well established compression standards. Our main motivation was to derive a biologically justified (i.e. a learning method) which is able to compete with well known numerical approaches to PCA.

RLS learning rate in matrix form for PSA and ordered PCA methods

We also have introduced a matrix form of the learning rate and its adaptation according to the RLS scheme for the PSA (principal subspace analysis) and ordered PCA (e.g. the Brockett algorithm). We obtained fast learning algorithms, that are able to learn an approximation of highest range components or subspaces with reasonable quality.

See our paper published at ESANN 1996.

As these methods are assumed to work in parallel manner, the signal deflation idea can not be applied straightforward to them. But without deflation these algorithms are not able properly to extract the whole range of PC's or PS's, and thus they are not competitive with our CRLS PCA method.
[go back to top],

W. Kasprzak
Apr. 1996