—We propose a new subspace clustering method that integrates feature selection into subspace clus... more —We propose a new subspace clustering method that integrates feature selection into subspace clustering. Rather than using all features to construct a low-rank representation of the data, we find such a representation using only relevant features, which helps in revealing more accurate data relationships. Two variants are proposed by using both convex and nonconvex rank approximations. Extensive experimental results confirm the effectiveness of the proposed method and models.
The importance of accurate recommender systems has
been widely recognized by academia and industr... more The importance of accurate recommender systems has been widely recognized by academia and industry. How- ever, the recommendation quality is still rather low. Recently, a linear sparse and low-rank representation of the user-item matrix has been applied to produce Top-N recommendations. This approach uses the nu- clear norm as a convex relaxation for the rank func- tion and has achieved better recommendation accuracy than the state-of-the-art methods. In the past several years, solving rank minimization problems by leveraging nonconvex relaxations has received increasing attention. Some empirical results demonstrate that it can provide a better approximation to original problems than con- vex relaxation. In this paper, we propose a novel rank approximation to enhance the performance of Top-N recommendation systems, where the approximation er- ror is controllable. Experimental results on real data show that the proposed rank approximation improves the Top-N recommendation accuracy substantially.
Top-N recommender systems have been investigated widely
both in industry and academia. However, t... more Top-N recommender systems have been investigated widely both in industry and academia. However, the recommendation quality is far from satisfactory. In this paper, we propose a simple yet promising algorithm. We fill the user-item matrix based on a low-rank assumption and simultaneously keep the original information. To do that, a nonconvex rank relaxation rather than the nuclear norm is adopted to provide a better rank approximation and an efficient optimization strategy is designed. A comprehensive set of experiments on real datasets demonstrates that our method pushes the accuracy of Top-N recommendation to a new level.
Low-rank matrix is desired in many machine learning and computer vision problems. Most of the rec... more Low-rank matrix is desired in many machine learning and computer vision problems. Most of the recent studies use the nuclear norm as a convex surrogate of the rank operator. However, all singular values are simply added together by the nuclear norm, and thus the rank may not be well approximated in practical problems. In this paper, we propose using a log-determinant (LogDet) function as a smooth and closer, though nonconvex, approximation to rank for obtaining a low-rank representation in subspace clustering. Augmented Lagrange multipliers strategy is applied to iteratively optimize the LogDet-based nonconvex objective function on potentially large-scale data. By making use of the angular information of principal directions of the resultant low- rank representation, an affinity graph matrix is constructed for spectral clustering. Experimental results on motion segmentation and face clustering data demonstrate that the proposed method often outperforms state-of-the-art subspace clustering algorithms.
A number of machine learning and computer vision problems, such as matrix
completion and subspace... more A number of machine learning and computer vision problems, such as matrix completion and subspace clustering, require a matrix to be of low-rank. To meet this requirement, most existing methods use the nuclear norm as a convex proxy of the rank function and minimize it. However, the nuclear norm simply adds all nonzero singular values together instead of treating them equally as the rank function does, which may not be a good rank approximation when some singular values are very large. To reduce this undesirable ...
Numerous applications in data mining and machine learning require recovering a matrix of mini... more Numerous applications in data mining and machine learning require recovering a matrix of minimal rank. Robust principal component analysis (RPCA) is a general framework for handling this kind of problems. Nuclear norm based convex surrogate of the rank function in RPCA is widely investigated. Under certain assumptions, it can recover the underlying true low rank matrix with high probability. However, those assumptions may not hold in real-world applications. Since the nuclear norm approximates the rank by adding all singular values together, which is essentially a ℓ1-norm of the singular values, the resulting approximation error is not trivial and thus the resulting matrix estimator can be significantly biased. To seek a closer approximation and to alleviate the above-mentioned limitations of the nuclear norm, we propose a nonconvex rank approximation. This approximation to the matrix rank is tighter than the nuclear norm. To solve the associated nonconvex minimization problem, we develop an efficient augmented Lagrange multiplier based optimization algorithm. Experimental results demonstrate that our method outperforms current state-of-the-art algorithms in both accuracy and efficiency.
Matrix rank minimizing subject to affine constraints arises in many application areas, ranging fr... more Matrix rank minimizing subject to affine constraints arises in many application areas, ranging from signal processing to machine learning. Nuclear norm is a convex relaxation for this problem which can recover the rank exactly under some restricted and theoretically interesting conditions. However, for many real-world applications, nuclear norm approximation to the rank function can only produce a result far from the optimum. To seek a solution of higher accuracy than the nuclear norm, in this letter, we propose a rank approximation based on Logarithm-Determinant. We consider using this rank approximation for subspace clustering application. Our framework can model different kinds of errors and noise. Effective optimization strategy is developed with theoretical guarantee to converge to a stationary point. The proposed method gives promising results on face clustering and motion segmentation tasks compared to the state-of-the-art subspace clustering algorithms.
Matrix rank minimization problem is in general NP-hard.
The nuclear norm is used to ... more Matrix rank minimization problem is in general NP-hard. The nuclear norm is used to substitute the rank function in many recent studies. Nevertheless, the nuclear norm ap- proximation adds all singular values together and the ap- proximation error may depend heavily on the magnitudes of singular values. This might restrict its capability in dealing with many practical problems. In this paper, an arctan- gent function is used as a tighter approximation to the rank function. We use it on the challenging subspace clustering problem. For this nonconvex minimization problem, we de- velop an effective optimization procedure based on a type of augmented Lagrange multipliers (ALM) method. Exten- sive experiments on face clustering and motion segmentation show that the proposed method is effective for rank approx- imation.
We demonstrate very high resolution photon spectroscopy with a microwave-multiplexed two-pixel tr... more We demonstrate very high resolution photon spectroscopy with a microwave-multiplexed two-pixel transition-edge sensor (TES) array. We measured a 153 Gd photon source and achieved an energy resolution of 63 eV full-width-at-half-maximum at 97 keV and an equivalent readout system noise of 86 pA= ffiffiffiffiffiffi Hz p at the TES. The readout circuit consists of superconducting microwave resonators coupled to radio-frequency superconducting-quantum-interference-devices and transduces changes in input current to changes in phase of a microwave signal. We use flux-ramp modulation to linearize the response and evade low-frequency noise. This demonstration establishes one path for the readout of cryogenic X-ray and gamma-ray sensor arrays with more than 10 3 elements and spectral resolving powers R ¼ k=Dk > 10 3 . V C 2013 AIP Publishing LLC.
—We propose a new subspace clustering method that integrates feature selection into subspace clus... more —We propose a new subspace clustering method that integrates feature selection into subspace clustering. Rather than using all features to construct a low-rank representation of the data, we find such a representation using only relevant features, which helps in revealing more accurate data relationships. Two variants are proposed by using both convex and nonconvex rank approximations. Extensive experimental results confirm the effectiveness of the proposed method and models.
The importance of accurate recommender systems has
been widely recognized by academia and industr... more The importance of accurate recommender systems has been widely recognized by academia and industry. How- ever, the recommendation quality is still rather low. Recently, a linear sparse and low-rank representation of the user-item matrix has been applied to produce Top-N recommendations. This approach uses the nu- clear norm as a convex relaxation for the rank func- tion and has achieved better recommendation accuracy than the state-of-the-art methods. In the past several years, solving rank minimization problems by leveraging nonconvex relaxations has received increasing attention. Some empirical results demonstrate that it can provide a better approximation to original problems than con- vex relaxation. In this paper, we propose a novel rank approximation to enhance the performance of Top-N recommendation systems, where the approximation er- ror is controllable. Experimental results on real data show that the proposed rank approximation improves the Top-N recommendation accuracy substantially.
Top-N recommender systems have been investigated widely
both in industry and academia. However, t... more Top-N recommender systems have been investigated widely both in industry and academia. However, the recommendation quality is far from satisfactory. In this paper, we propose a simple yet promising algorithm. We fill the user-item matrix based on a low-rank assumption and simultaneously keep the original information. To do that, a nonconvex rank relaxation rather than the nuclear norm is adopted to provide a better rank approximation and an efficient optimization strategy is designed. A comprehensive set of experiments on real datasets demonstrates that our method pushes the accuracy of Top-N recommendation to a new level.
Low-rank matrix is desired in many machine learning and computer vision problems. Most of the rec... more Low-rank matrix is desired in many machine learning and computer vision problems. Most of the recent studies use the nuclear norm as a convex surrogate of the rank operator. However, all singular values are simply added together by the nuclear norm, and thus the rank may not be well approximated in practical problems. In this paper, we propose using a log-determinant (LogDet) function as a smooth and closer, though nonconvex, approximation to rank for obtaining a low-rank representation in subspace clustering. Augmented Lagrange multipliers strategy is applied to iteratively optimize the LogDet-based nonconvex objective function on potentially large-scale data. By making use of the angular information of principal directions of the resultant low- rank representation, an affinity graph matrix is constructed for spectral clustering. Experimental results on motion segmentation and face clustering data demonstrate that the proposed method often outperforms state-of-the-art subspace clustering algorithms.
A number of machine learning and computer vision problems, such as matrix
completion and subspace... more A number of machine learning and computer vision problems, such as matrix completion and subspace clustering, require a matrix to be of low-rank. To meet this requirement, most existing methods use the nuclear norm as a convex proxy of the rank function and minimize it. However, the nuclear norm simply adds all nonzero singular values together instead of treating them equally as the rank function does, which may not be a good rank approximation when some singular values are very large. To reduce this undesirable ...
Numerous applications in data mining and machine learning require recovering a matrix of mini... more Numerous applications in data mining and machine learning require recovering a matrix of minimal rank. Robust principal component analysis (RPCA) is a general framework for handling this kind of problems. Nuclear norm based convex surrogate of the rank function in RPCA is widely investigated. Under certain assumptions, it can recover the underlying true low rank matrix with high probability. However, those assumptions may not hold in real-world applications. Since the nuclear norm approximates the rank by adding all singular values together, which is essentially a ℓ1-norm of the singular values, the resulting approximation error is not trivial and thus the resulting matrix estimator can be significantly biased. To seek a closer approximation and to alleviate the above-mentioned limitations of the nuclear norm, we propose a nonconvex rank approximation. This approximation to the matrix rank is tighter than the nuclear norm. To solve the associated nonconvex minimization problem, we develop an efficient augmented Lagrange multiplier based optimization algorithm. Experimental results demonstrate that our method outperforms current state-of-the-art algorithms in both accuracy and efficiency.
Matrix rank minimizing subject to affine constraints arises in many application areas, ranging fr... more Matrix rank minimizing subject to affine constraints arises in many application areas, ranging from signal processing to machine learning. Nuclear norm is a convex relaxation for this problem which can recover the rank exactly under some restricted and theoretically interesting conditions. However, for many real-world applications, nuclear norm approximation to the rank function can only produce a result far from the optimum. To seek a solution of higher accuracy than the nuclear norm, in this letter, we propose a rank approximation based on Logarithm-Determinant. We consider using this rank approximation for subspace clustering application. Our framework can model different kinds of errors and noise. Effective optimization strategy is developed with theoretical guarantee to converge to a stationary point. The proposed method gives promising results on face clustering and motion segmentation tasks compared to the state-of-the-art subspace clustering algorithms.
Matrix rank minimization problem is in general NP-hard.
The nuclear norm is used to ... more Matrix rank minimization problem is in general NP-hard. The nuclear norm is used to substitute the rank function in many recent studies. Nevertheless, the nuclear norm ap- proximation adds all singular values together and the ap- proximation error may depend heavily on the magnitudes of singular values. This might restrict its capability in dealing with many practical problems. In this paper, an arctan- gent function is used as a tighter approximation to the rank function. We use it on the challenging subspace clustering problem. For this nonconvex minimization problem, we de- velop an effective optimization procedure based on a type of augmented Lagrange multipliers (ALM) method. Exten- sive experiments on face clustering and motion segmentation show that the proposed method is effective for rank approx- imation.
We demonstrate very high resolution photon spectroscopy with a microwave-multiplexed two-pixel tr... more We demonstrate very high resolution photon spectroscopy with a microwave-multiplexed two-pixel transition-edge sensor (TES) array. We measured a 153 Gd photon source and achieved an energy resolution of 63 eV full-width-at-half-maximum at 97 keV and an equivalent readout system noise of 86 pA= ffiffiffiffiffiffi Hz p at the TES. The readout circuit consists of superconducting microwave resonators coupled to radio-frequency superconducting-quantum-interference-devices and transduces changes in input current to changes in phase of a microwave signal. We use flux-ramp modulation to linearize the response and evade low-frequency noise. This demonstration establishes one path for the readout of cryogenic X-ray and gamma-ray sensor arrays with more than 10 3 elements and spectral resolving powers R ¼ k=Dk > 10 3 . V C 2013 AIP Publishing LLC.
Uploads
Papers by Zhao Kang
been widely recognized by academia and industry. How-
ever, the recommendation quality is still rather low.
Recently, a linear sparse and low-rank representation
of the user-item matrix has been applied to produce
Top-N recommendations. This approach uses the nu-
clear norm as a convex relaxation for the rank func-
tion and has achieved better recommendation accuracy
than the state-of-the-art methods. In the past several
years, solving rank minimization problems by leveraging
nonconvex relaxations has received increasing attention.
Some empirical results demonstrate that it can provide
a better approximation to original problems than con-
vex relaxation. In this paper, we propose a novel rank
approximation to enhance the performance of Top-N
recommendation systems, where the approximation er-
ror is controllable. Experimental results on real data
show that the proposed rank approximation improves
the Top-N recommendation accuracy substantially.
both in industry and academia. However, the recommendation
quality is far from satisfactory. In this paper, we propose
a simple yet promising algorithm. We fill the user-item matrix
based on a low-rank assumption and simultaneously keep
the original information. To do that, a nonconvex rank relaxation
rather than the nuclear norm is adopted to provide a
better rank approximation and an efficient optimization strategy
is designed. A comprehensive set of experiments on real
datasets demonstrates that our method pushes the accuracy of
Top-N recommendation to a new level.
norm as a convex surrogate of the rank operator. However, all singular values are simply added together by the nuclear norm, and
thus the rank may not be well approximated in practical problems. In this paper, we propose using a log-determinant (LogDet)
function as a smooth and closer, though nonconvex, approximation to rank for obtaining a low-rank representation in subspace
clustering. Augmented Lagrange multipliers strategy is applied to iteratively optimize the LogDet-based nonconvex objective
function on potentially large-scale data. By making use of the angular information of principal directions of the resultant low-
rank representation, an affinity graph matrix is constructed for spectral clustering. Experimental results on motion segmentation
and face clustering data demonstrate that the proposed method often outperforms state-of-the-art subspace clustering algorithms.
completion and subspace clustering, require a matrix to be of low-rank. To meet this
requirement, most existing methods use the nuclear norm as a convex proxy of the rank
function and minimize it. However, the nuclear norm simply adds all nonzero singular values
together instead of treating them equally as the rank function does, which may not be a good
rank approximation when some singular values are very large. To reduce this undesirable ...
The nuclear norm is used to substitute the rank function
in many recent studies. Nevertheless, the nuclear norm ap-
proximation adds all singular values together and the ap-
proximation error may depend heavily on the magnitudes of
singular values. This might restrict its capability in dealing
with many practical problems. In this paper, an arctan-
gent function is used as a tighter approximation to the rank
function. We use it on the challenging subspace clustering
problem. For this nonconvex minimization problem, we de-
velop an effective optimization procedure based on a type
of augmented Lagrange multipliers (ALM) method. Exten-
sive experiments on face clustering and motion segmentation
show that the proposed method is effective for rank approx-
imation.
been widely recognized by academia and industry. How-
ever, the recommendation quality is still rather low.
Recently, a linear sparse and low-rank representation
of the user-item matrix has been applied to produce
Top-N recommendations. This approach uses the nu-
clear norm as a convex relaxation for the rank func-
tion and has achieved better recommendation accuracy
than the state-of-the-art methods. In the past several
years, solving rank minimization problems by leveraging
nonconvex relaxations has received increasing attention.
Some empirical results demonstrate that it can provide
a better approximation to original problems than con-
vex relaxation. In this paper, we propose a novel rank
approximation to enhance the performance of Top-N
recommendation systems, where the approximation er-
ror is controllable. Experimental results on real data
show that the proposed rank approximation improves
the Top-N recommendation accuracy substantially.
both in industry and academia. However, the recommendation
quality is far from satisfactory. In this paper, we propose
a simple yet promising algorithm. We fill the user-item matrix
based on a low-rank assumption and simultaneously keep
the original information. To do that, a nonconvex rank relaxation
rather than the nuclear norm is adopted to provide a
better rank approximation and an efficient optimization strategy
is designed. A comprehensive set of experiments on real
datasets demonstrates that our method pushes the accuracy of
Top-N recommendation to a new level.
norm as a convex surrogate of the rank operator. However, all singular values are simply added together by the nuclear norm, and
thus the rank may not be well approximated in practical problems. In this paper, we propose using a log-determinant (LogDet)
function as a smooth and closer, though nonconvex, approximation to rank for obtaining a low-rank representation in subspace
clustering. Augmented Lagrange multipliers strategy is applied to iteratively optimize the LogDet-based nonconvex objective
function on potentially large-scale data. By making use of the angular information of principal directions of the resultant low-
rank representation, an affinity graph matrix is constructed for spectral clustering. Experimental results on motion segmentation
and face clustering data demonstrate that the proposed method often outperforms state-of-the-art subspace clustering algorithms.
completion and subspace clustering, require a matrix to be of low-rank. To meet this
requirement, most existing methods use the nuclear norm as a convex proxy of the rank
function and minimize it. However, the nuclear norm simply adds all nonzero singular values
together instead of treating them equally as the rank function does, which may not be a good
rank approximation when some singular values are very large. To reduce this undesirable ...
The nuclear norm is used to substitute the rank function
in many recent studies. Nevertheless, the nuclear norm ap-
proximation adds all singular values together and the ap-
proximation error may depend heavily on the magnitudes of
singular values. This might restrict its capability in dealing
with many practical problems. In this paper, an arctan-
gent function is used as a tighter approximation to the rank
function. We use it on the challenging subspace clustering
problem. For this nonconvex minimization problem, we de-
velop an effective optimization procedure based on a type
of augmented Lagrange multipliers (ALM) method. Exten-
sive experiments on face clustering and motion segmentation
show that the proposed method is effective for rank approx-
imation.