Subspace Sequentially Iterative Leaning for Semi-Supervised SVM

Classifying partially labeled high-dimensional data remains a difficult problem for semi-supervised support vector machine (SVM) since the convergence and the stability can hardly be guaranteed. Existing studies try to use dimensionality reduction techniques to relieve this problem. But the extracted features may not be suitable for the downstream classifier, leading to a sub-optimal classification performance. To address these problems, this paper proposes a novel semi-supervised framework named subspace sequentially Iterative SVM (ISVM) to integrate semi-supervised learning, high-dimensional data processing, and classifier learning into a unified framework. That is, ISVM expects to learn an optimal subspace by trading off multiple factors, including joint sparsity, regression learning, Laplacian graph regularization, and semi-supervised support vector learning, to provide a large margin for semi-supervised SVM classifier. The proposed framework not only owns the merits of subspace learning to solve dimensional disaster problem and large-scale data problem, but also has an effective mechanism to optimize different tasks perfectly. Theoretical analysis shows that the optimal solution to the original problem can be given by solving its dual problem, and the convergence of the optimization process can be guaranteed by the Karush-Kuhn-Tucker(KKT) conditions. Extensive experiments have been performed on some well-known datasets to validate the superiority of the proposed ISVM compared with the state-of-the-art algorithms.