Robust Covariance Representations With Large Margin Dimensionality Reduction for Visual Classification

Inspired by the breakthrough performance of deep convolutional neural networks (CNNs) and the effectiveness of covariance representations, the combination of covariances with activations of deep CNNs has great potential in representing visual concepts. However, such method lies in two challenges: 1) robust estimation of covariance in the case of high dimension and small sample size and 2) high computational and storage costs caused by high-dimensional covariance representations. To tackle the above challenges, this paper proposes a novel robust covariance representation with large-margin dimensionality reduction for visual classification. First, we introduce two regularized maximum likelihood estimators to perform the robust estimation of covariance in the case of high dimension and small sample size, which can greatly improve the modeling ability of covariances. Then, we present a large-margin dimensionality reduction method for high-dimensional covariance representations. It does not only significantly reduce the dimension of robust covariance representations with considering their Riemannian geometry structure, but also can further enhance their discriminability. Experiments are conducted on three kinds of visual classification tasks, and the results show that our proposed method is superior to its counterparts and achieves the state-of-the-art performance.