Deep Inductive and Scalable Subspace Clustering via Nonlocal Contrastive Self-distillation
Date
Authors
Zhu, W
Peng, B
Yan, WeiQi
Supervisor
Item type
Journal Article
Degree name
Journal Title
Journal ISSN
Volume Title
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Abstract
Deep subspace clustering has demonstrated remarkable results by leveraging the nonlinear subspace assumption. However, it often encounters challenges in terms of computational cost and memory footprint in dealing with large-scale data due to its traditional single-batch training strategy. To address this issue, this paper proposes a deep subspace clustering framework that is regularized by nonlocal contrastive self-distillation, enabling a Deep Inductive and Scalable Subspace Clustering (DISSC) algorithm. In particular, our framework incorporates two subspace learning modules, namely subspace learning based on self-expression model and inductive subspace clustering. These modules generate affinities from different perspectives by extracting intermediate features from two augmentations of the input data using a weight-sharing neural network. By integrating the concept of self-distillation, our framework effectively exploits the clustering-friendly knowledge contained in these two affinities through a novel nonlocal contrastive prediction task, employing an empirical yet effective threshold. This allows the framework to facilitate complementary knowledge mining and scalability without compromising clustering performance. With an alternate branch that bypasses the self-expression computation, our framework can infer subspace membership of the out-of-sample data through the predicted soft labels, eliminating the need for ad-hoc postprocessing. In addition, the self-expression matrix computed using mini-batch data benefits from the distilled knowledge obtained from the inductive subspace clustering module, enabling our framework to scale to data of arbitrary size. Experiments conducted on large-scale MNIST, Fashion-MINST, STL-10, CIFAR-10 and Stanford Online Products datasets validate the superiority of the proposed DISSC algorithm over state-of-the-art subspace clustering methods.Description
Keywords
46 Information and Computing Sciences, 4611 Machine Learning, Bioengineering, Networking and Information Technology R&D (NITRD), Machine Learning and Artificial Intelligence, 0801 Artificial Intelligence and Image Processing, 0906 Electrical and Electronic Engineering, Artificial Intelligence & Image Processing, 4006 Communications engineering, 4009 Electronics, sensors and digital hardware, 4603 Computer vision and multimedia computation
Source
IEEE Transactions on Circuits and Systems for Video Technology, ISSN: 1051-8215 (Print); 1558-2205 (Online), Institute of Electrical and Electronics Engineers (IEEE), PP(99), 1-1. doi: 10.1109/TCSVT.2025.3613980
Publisher's version
Rights statement
This is the Author's Accepted Manuscript of an article published in IEEE Transactions on Circuits and Systems for Video Technology. The Version of Record will be available at DOI: 10.1109/TCSVT.2025.3613980