Principal
Component Null Space Analysis (PCNSA) for Image, Video (or any large
dimensional pattern) Classification
We present a new classification algorithm, Principal Component Null Space Analysis (PCNSA), which is designed for classification problems like object recognition where different classes have unequal and non-white noise covariance matrices. PCNSA first obtains a principal components’ subspace (PCA space) for the entire data. In this PCA space, it finds for each class ‘i’, an M_i dimensional subspace along which the class’s intra-class variance is the smallest. We call this subspace an Approximate Null Space (ANS) since the lowest variance is usually “much smaller” than the highest. A query is classified into class ‘i’ if its distance from the class’s mean in the class’s ANS is a minimum.
We derive upper bounds on classification error probability of PCNSA and use these expressions to compare classification performance of PCNSA with that of Subspace Linear Discriminant Analysis (SLDA). We propose a practical modification of PCNSA called progressive-PCNSA that also detects ‘new’ (untrained classes).
We provide an experimental comparison of PCNSA and progressive-PCNSA with SLDA and PCA and also with other classification algorithms - linear SVMs, kernel PCA, kernel discriminant analysis and kernel SLDA, for object recognition and face recognition under large pose/expression variation. We also show application of PCNSA to two classification problems in video - an action retrieval problem and abnormal activity detection.