Subspace Method

Summary

The Subspace Method (SM) is a classification technique fundamental to all other subspace based methods. There are many extensions of SM such as MSM, CMSM, OMSM and so on.

Fig.1

Note

Check out the tutorial, “from pca to the subspace method” for a detailed walkthrough for obtaining subspaces using pca.

Theory

The Subspace method assumes an input vector \(\mathbf{x}'\) and \(k\)-class subspaces. Each class subspace approximates a data distribution for a single class. This approximation is obtained by applying PCA to each class [Fuk14].

The similarity \(S\) of the input vector \(\mathbf{x}'\) to the \(i^{th}\) class subspace \(Q_i\) is defined based on either:

• The length of the projection of \(\mathbf{x}'\) to \(Q_i\) [Wat67].

• The minimum angle between \(\mathbf{x}'\) and \(Q_i\) [IGM74].

The length of an input vector \(\mathbf{x}'\) is often normalized to 1.0. In this case these two criteria are identical. This fact should be obvious from Fig.1.

Since they are the same, from here on we think of the angle-based similarity \(S\) defined by the following equation:

\[S = \cos{\theta} = \sum^{k}_{i=1} \frac{(\mathbf{x}' · \Phi_i)^2}{||\mathbf{x}'||^2}\]

\(\Phi_i\) is the \(i^{th}\) orthogonal normal basis vector of the class subspace \(Q_i\), which are obtained from applying the principal component analysis (PCA) to a set of patterns of the class. In more rigourus terms, these orthonormal basis vectors can be obtained as the eigenvectors of the correlation matrix

\[\sum^{l}_{i=1}\mathbf{x}^{(i)}\mathbf{x}^{(i)\top} where \mathbf{x}^{(i)} \in \mathbb{X}\]

of the class (\(\mathbb{X}\) is the training dataset).

Learning Phase

1. Generate \(k\) class subspaces from each class by using PCA.

Recognition Phase

1. Calculate \(S\) between \(\mathbf{x}\) and each subspace \(Q_i\).

2. Classify the \(\mathbf{x}\) into the class where \(S\) was calculated to be the highest.

Also, you may add a rejection thershold on \(S\) to reject classifications with low similarity.