Unsupervised feature selection through preservation of locality relation

Abstract

Unsupervised feature selection (UFS) aims to find the most useful features from unlabeled data, and it remains an important research topic in machine learning. To preserve the data manifold, many UFS methods have been proposed. These methods utilize various techniques such as similarity preservation, spectral learning, sparse modeling, and matrix decomposition. However, many existing approaches encounter various challenges because they rely on explicit pairwise distance computations, are sensitive to noise, and suffer from problems arising from high dimensionality. Besides, they have limited capability in preserving the manifold structure, especially when neighborhood graphs are heuristically defined or when embeddings distort the geometry. To overcome these issues, we propose a novel unsupervised feature selection framework that uses a geometry-aware optimization model. The main purpose is to learn a diagonal transformation matrix that captures the features' importance while maintaining local and global manifold structures. While carrying out this process, we do not directly calculate pairwise distances. This strategy increasescomputational robustness. The diagonal structure enables interpretable feature suppression by scaling or collapsing dimensions. This feature suppression leads to better generalization. The results of the benchmark data sets indicates that the proposed approach provides a convenient trade-off between structure preservation and the effectiveness of feature selection, and the proposed method often achieves performance comparable to or better than six state-of-the-art UFS methods.