A Local SVD Framework for Stable Feature Selection for Clustering

Feature selection for clustering is a challenging problem due to the absence of class labels. Existing approaches can select a feature subset to maintain clustering performance while reducing dimensionality. However, we are faced with two problems: (1) there could be many sets of features that seem equally good, and (2) these features are sensitive to small data perturbation, or the selection instability problem. In this work, we investigate the stability problem in feature selection for clustering. To the best of our knowledge, this is the first work that aims to improve the stability of feature selection algorithms for clustering. The importance comes from the fact that stable selection provides consistent meaning for clusters. In this paper, we first formally define the problem and propose a Local Singular Value Decomposition (LSVD) framework for stable and accurate feature selection. Empirical results on various data sets show that the proposed framework can significantly improve selection stability whilst maintaining the clustering performance comparing to the baseline methods. An additional advantage of this approach is that the selected features preserve the physical meaning of the original features, a desirable property for subsequent data analysis.