Principal Component Null Space Analysis (PCNSA) for Image, Video (or any large dimensional pattern) Classification

We present a new classification algorithm, Principal Component Null Space Analysis (PCNSA), which is designed for classification problems like object recognition where different classes have unequal and non-white noise covariance matrices. PCNSA first obtains a principal components’ subspace (PCA space) for the entire data. In this PCA space, it finds for each class ‘i’, an M_i dimensional subspace along which the class’s intra-class variance is the smallest. We call this subspace an Approximate Null Space (ANS) since the lowest variance is usually “much smaller” than the highest. A query is classified into class ‘i’ if its distance from the class’s mean in the class’s ANS is a minimum.

We derive upper bounds on classification error probability of PCNSA and use these expressions to compare classification performance of PCNSA with that of Subspace Linear Discriminant Analysis (SLDA). We propose a practical modification of PCNSA called progressive-PCNSA that also detects ‘new’ (untrained classes).

We provide an experimental comparison of PCNSA and progressive-PCNSA with SLDA and PCA and also with other classification algorithms - linear SVMs, kernel PCA, kernel discriminant analysis and kernel SLDA, for object recognition and face recognition under large pose/expression variation. We also show application of PCNSA to two classification problems in video - an action retrieval problem and abnormal activity detection.


Papers
  • Namrata Vaswani, Rama Chellappa, " Principal Component Null Space Analysis for Image and Video Classification", to appear IEEE Trans. Image Processing, July 2006.
  • Namrata Vaswani, Rama Chellappa, "Classification Probability Analysis of Principal Component Null Space Analysis",   Intl. Conference on  Pattern Recognition (ICPR), 2004.
  • Namrata Vaswani,  "A  Linear Classifier for Gaussian Class Conditional Distributions with Unequal Covariance Matrices",  Intl. Conference on  Pattern Recognition (ICPR), 2002.

  • MATLAB code: pcnsa.zip 
    Unzip this file. There is a README.txt file inside it, which has instructions on how to use the code. Also there is a mat file datavecs.mat, which has the COIL object database images which can be used to compare PCNSA and Prog PCNSA against PCA, LDA, and kernel methods.