Skip to main content

Seungjin Choi

Followers

6

Following

1

Public Views

Mariya Ivancheva

University of Strathclyde, Glasgow

ESADE

Microsoft Research

Radhika Gajjala

Bowling Green State University

University of Cologne

Armando Marques-Guedes

UNL - New University of Lisbon

University of Gaziantep

Ronald E Goldsmith

Florida State University

DR. RAKESH KUMAR

University of Delhi

Anthony Patterson

Lancaster University Management School

Interests

Uploads

Papers by Seungjin Choi

Bayesian Hierarchical Clustering with Exponential Family: Small-Variance Asymptotics and Reducibility

arXiv (Cornell University), Jan 29, 2015

Bayesian hierarchical clustering (BHC) is an agglomerative clustering method, where a probabilist... more Bayesian hierarchical clustering (BHC) is an agglomerative clustering method, where a probabilistic model is defined and its marginal likelihoods are evaluated to decide which clusters to merge. While BHC provides a few advantages over traditional distance-based agglomerative clustering algorithms, successive evaluation of marginal likelihoods and careful hyperparameter tuning are cumbersome and limit the scalability. In this paper we relax BHC into a non-probabilistic formulation, exploring smallvariance asymptotics in conjugate-exponential models. We develop a novel clustering algorithm, referred to as relaxed BHC (RBHC), from the asymptotic limit of the BHC model that exhibits the scalability of distance-based agglomerative clustering algorithms as well as the flexibility of Bayesian nonparametric models. We also investigate the reducibility of the dissimilarity measure emerged from the asymptotic limit of the BHC model, allowing us to use scalable algorithms such as the nearest neighbor chain algorithm. Numerical experiments on both synthetic and real-world datasets demonstrate the validity and high performance of our method.

On Uncertainty Estimation by Tree-based Surrogate Models in Sequential Model-based Optimization

arXiv (Cornell University), Feb 21, 2022

Sequential model-based optimization sequentially selects a candidate point by constructing a surr... more Sequential model-based optimization sequentially selects a candidate point by constructing a surrogate model with the history of evaluations, to solve a black-box optimization problem. Gaussian process (GP) regression is a popular choice as a surrogate model, because of its capability of calculating prediction uncertainty analytically. On the other hand, an ensemble of randomized trees is another option and has practical merits over GPs due to its scalability and easiness of handling continuous/discrete mixed variables. In this paper we revisit various ensembles of randomized trees to investigate their behavior in the perspective of prediction uncertainty estimation. Then, we propose a new way of constructing an ensemble of randomized trees, referred to as BwO forest, where bagging with oversampling is employed to construct bootstrapped samples that are used to build randomized trees with random splitting. Experimental results demonstrate the validity and good performance of BwO forest over existing tree-based models in various circumstances.

Blind separation of nonstationary and temporally correlated sources from noisy mixtures

Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501)

Local Stability Analysis Of Flexible Independent Component Analysis Algorithm

Acoustics, Speech, and Signal Processing, 1988. ICASSP-88., 1988 International Conference on

This paper addresses local stability analysis for the exible independent component analysis (ICA)... more This paper addresses local stability analysis for the exible independent component analysis (ICA) algorithm 6] where the generalized Gaussian density m o d e l w as employed for blind separation of mixtures of sub-and super-Gaussian sources. In the exible ICA algorithm, the shape of nonlinear function in the learning algorithm varies depending on the Gaussian exponent which is properly selected according to the kurtosis of estimated source. In the framework of the natural gradient in Stiefel manifold, the exible ICA algorithm is revisited and some new results about its local stability analysis are presented.

Topographic Independent Component Analysis of Gene Expression Time Series Data

Topographic independent component analysis (TICA) is an interesting extension of the conventional... more Topographic independent component analysis (TICA) is an interesting extension of the conventional ICA, which aims at finding a linear decomposition into approximately independent components with the dependence between two components is approximated by their proximity in the topographic representation. In this paper we apply the topographic ICA to gene expression time series data and compare it with the conventional ICA as well as the independent subspace analysis (ISA). Empirical study with yeast cell cycle-related data and yeast sporulation data, shows that TICA is more suitable for gene clustering.

Iterative Projection Approximation Algorithms for PCA

2006 IEEE International Conference on Acoustics Speed and Signal Processing Proceedings

In this paper we introduce a new error measure, integrated reconstruction error (IRE), the minimi... more In this paper we introduce a new error measure, integrated reconstruction error (IRE), the minimization of which leads to principal eigenvectors (without rotational ambiguity) of the data covariance matrix. Then we present iterative algorithms for the IRE minimization, through the projection approximation. The proposed algorithm is referred to as COnstrained Projection Approximation (COPA) algorithm and its limiting case is called COPAL. We also discuss regularized algorithms, referred to as R-COPA and R-COPAL. Numerical experiments demonstrate that these algorithms successfully find exact principal eigenvectors of the data covariance matrix.

Bayesian Multi-task Learning for Common Spatial Patterns

2011 International Workshop on Pattern Recognition in NeuroImaging, 2011

Common spatial pattern (CSP) is a widely-used feature extraction method for electroencephalogram ... more Common spatial pattern (CSP) is a widely-used feature extraction method for electroencephalogram (EEG) classification and corresponding probabilistic models were recently developed, adopting a linear generative model for each class. These models are trained on a subject-by-subject basis so that inter-subject information is neglected. Moreover when only a few training samples are available for each subject, the performance is degraded. In this paper we employ Bayesian multi-task learning so that subject-to-subject information is transferred in learning the model for a subject of interest. We present two probabilistic models where precision parameters of multivariate or matrix-variate Gaussian prior for the dictionary are shared across subjects. Numerical experiments on the BCI competition IV 2a dataset confirm that our methods improve classification performance over the standard CSP (on a subject-by-subject basis), especially in the case of subjects with fewer number of training samples.

Sequential Spectral Learning to Hash with Multiple Representations

Computer Vision – ECCV 2012, 2012

Learning to hash involves learning hash functions from a set of images for embedding high-dimensi... more Learning to hash involves learning hash functions from a set of images for embedding high-dimensional visual descriptors into a similarity-preserving low-dimensional Hamming space. Most of existing methods resort to a single representation of images, that is, only one type of visual descriptors is used to learn a hash function to assign binary codes to images. However, images are often described by multiple different visual descriptors (such as SIFT, GIST, HOG), so it is desirable to incorporate these multiple representations into learning a hash function, leading to multi-view hashing. In this paper we present a sequential spectral learning approach to multi-view hashing where a hash function is sequentially determined by solving the successive maximization of local variances subject to decorrelation constraints. We compute multi-view local variances by α-averaging view-specific distance matrices such that the best averaged distance matrix is determined by minimizing its α-divergence from view-specific distance matrices. We also present a scalable implementation, exploiting a fast approximate k-NN graph construction method, in which α-averaged distances computed in small partitions determined by recursive spectral bisection are gradually merged in conquer steps until whole examples are used. Numerical experiments on Caltech-256, CIFAR-20, and NUS-WIDE datasets confirm the high performance of our method, in comparison to single-view spectral hashing as well as existing multi-view hashing methods.

Hashing with Generalized Nyström Approximation

2012 IEEE 12th International Conference on Data Mining, 2012

Hashing, which involves learning binary codes to embed high-dimensional data into a similarity-pr... more Hashing, which involves learning binary codes to embed high-dimensional data into a similarity-preserving low-dimensional Hamming space, is often formulated as linear dimensionality reduction followed by binary quantization. Linear dimensionality reduction, based on maximum variance formulation, requires leading eigenvectors of data covariance or graph Laplacian matrix. Computing leading singular vectors or eigenvectors in the case of high-dimension and large sample size, is a main bottleneck in most of data-driven hashing methods. In this paper we address the use of generalized Nyström method where a subset of rows and columns are used to approximately compute leading singular vectors of the data matrix, in order to improve the scalability of hashing methods in the case of high-dimensional data with large sample size. Especially we validate the useful behavior of generalized Nyström approximation with uniform sampling, in the case of a recentlydeveloped hashing method based on principal component analysis (PCA) followed by an iterative quantization, referred to as PCA+ITQ, developed by Gong and Lazebnik. We compare the performance of generalized Nyström approximation with uniform and non-uniform sampling, to the full singular value decomposition (SVD) method, confirming that the uniform sampling improves the computational and space complexities dramatically, while the performance is not much sacrificed. In addition we present low-rank approximation error bounds for generalized Nyström approximation with uniform sampling, which is not a trivial extension of available results on the nonuniform sampling case.

Multi-view anchor graph hashing

2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 2013

Multi-view hashing seeks compact integrated binary codes which preserve similarities averaged ove... more Multi-view hashing seeks compact integrated binary codes which preserve similarities averaged over multiple representations of objects. Most of existing multi-view hashing methods resort to linear hash functions where data manifold is not considered. In this paper we present multi-view anchor graph hashing (MVAGH), where nonlinear integrated binary codes are efficiently determined by a subset of eigenvectors of an averaged similarity matrix. The efficiency behind MVAGH is due to a low-rank form of the averaged similarity matrix induced by multi-view anchor graph, where the similarity between two points is measured by two-step transition probability through view-specific anchor (i.e. landmark) points. In addition, we observe that MVAGH suffers from the performance degradation when the high recall is required. To overcome this drawback, we propose a simple heuristic to combine MVAGH with locality sensitive hashing (LSH). Numerical experiments on CIFAR-10 dataset confirms that MVAGH(+LSH) outperforms the existing multi-and single-view hashing methods.

Incremental Tree-Based Inference with Dependent Normalized Random Measures

Normalized random measures (NRMs) form a broad class of discrete random measures that are used as... more Normalized random measures (NRMs) form a broad class of discrete random measures that are used as priors for Bayesian nonparametric models. Dependent normalized random measures (DNRMs) introduce dependencies in a set of NRMs, to facilitate the handling of data where the assumption of exchangeability is violated. Various methods have been developed to construct DNRMs; of particular interest is mixed normalized random measures (MNRMs), where DNRM is represented as a mixture of underlying shared normalized random measures. Emphasis in existing works is placed on the construction methods of DNRMs, but there is a little work on efficient inference for DNRMs. In this paper, we present a tree-based inference method for MNRM mixture models, extending Bayesian hierarchical clustering (BHC) which was originally developed as a deterministic approximate inference for Dirichlet process mixture (DPM) models. We also present an incremental inference for MNRM mixture models, building a tree incrementally in the sense that the tree structure is partially updated whenever a new data point comes in. The tree, when constructed in such a way, allows us to efficiently perform tree-consistent MAP inference in MRNM mixture models, determining a most probable tree-consistent partition, as well as to compute a marginal likelihood approximately. Numerical experiments on both synthetic and real-world datasets demonstrate the usefulness of our algorithm, compared to MCMC methods.

Clustering sequence sets for motif discovery

Most of existing methods for DNA motif discovery consider only a single set of sequences to find ... more Most of existing methods for DNA motif discovery consider only a single set of sequences to find an over-represented motif. In contrast, we consider multiple sets of sequences where we group sets associated with the same motif into a cluster, assuming that each set involves a single motif. Clustering sets of sequences yields clusters of coherent motifs, improving signal-to-noise ratio or enabling us to identify multiple motifs. We present a probabilistic model for DNA motif discovery where we identify multiple motifs through searching for patterns which are shared across multiple sets of sequences. Our model infers cluster-indicating latent variables and learns motifs simultaneously, where these two tasks interact with each other. We show that our model can handle various motif discovery problems, depending on how to construct multiple sets of sequences. Experiments on three different problems for discovering DNA motifs emphasize the useful behavior and confirm the substantial gains over existing methods where only a single set of sequences is considered.

Self-labeling for P300 detection

2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2012

The P300 wave refers to a positive peak with a latency of 300 ms, produced in response to task-re... more The P300 wave refers to a positive peak with a latency of 300 ms, produced in response to task-relevant stimuli. It is a widely-used event related potential (ERP) in practical brain computer interface (BCI) systems, where a classifier is trained using discriminative features extracted from a set of labeled examples, in order to detect the presence of P300. Given a small training examples, the intra-and inter-subject variations in amplitude and latency of P300 degrade the performance of classifier. Thus, it requires a longer training period (calibration time) with more labeled examples for satisfactory performance. In this paper we present a self-labeling method, where confident unlabeled data, together with their predicted labels, are gradually added to the training set, in order to re-train the classifier. Linear discriminant analysis with singular value decomposition is used to progressively extract discriminate features. Experiments demonstrate the high performance of our method, especially in the case where a small number of training examples are available. We also apply the method to the zero-calibration P300-based BCI, which removes subject-dependent calibration procedures by using the training set already recorded from other subjects.

Learning α-integration with partially-labeled data

2010 IEEE International Conference on Acoustics, Speech and Signal Processing, 2010

Sensory data integration is an important task in human brain for multimodal processing as well as... more Sensory data integration is an important task in human brain for multimodal processing as well as in machine learning for multisensor processing. α-integration was proposed by Amari as a principled way of blending multiple positive measures (e.g., stochastic models in the form of probability distributions), providing an optimal integration in the sense of minimizing the α-divergence. It also encompasses existing integration methods as its special case, e.g., weighted average and exponential mixture. In α-integration, the value of α determines the characteristics of the integration and the weight vector w assigns the degree of importance to each measure. In most of the existing work, however, α and w are given in advance rather than learned. In this paper we present two algorithms, for learning α and w from data when only a few integrated target values are available. Numerical experiments on synthetic as well as real-world data confirm the proposed method's effectiveness.

Source Separation with Gaussian Process Models

Lecture Notes in Computer Science

Online multi-label learning with accelerated nonsmooth stochastic gradient descent

2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 2013

Rao-Blackwellized Particle Filtering for Sequential Speech Enhancement

The 2006 IEEE International Joint Conference on Neural Network Proceedings

In this paper we present a method of sequential speech enhancement, where we infer clean speech s... more In this paper we present a method of sequential speech enhancement, where we infer clean speech signal using a Rao-Blackwellized particle filter (RBPF), given a noisecontaminated observed signal. In contrast to Kalman filteringbased methods, we consider a non-Gaussian speech generative model that is based on the generalized auto-regressive (GAR) model. Model parameters are learned by sequential expectation maximization, incorporating the RBPF. Empirical comparison to Kalman filter, confirms the high performance of the proposed method.

Orthogonal Nonnegative Matrix Factorization for Blind Image Separation

Lecture Notes in Computer Science, 2013

Online Video Segmentation by Bayesian Split-Merge Clustering

Lecture Notes in Computer Science, 2012

We present an online video segmentation algorithm based on a novel nonparametric Bayesian cluster... more We present an online video segmentation algorithm based on a novel nonparametric Bayesian clustering method called Bayesian Split-Merge Clustering (BSMC). BSMC can efficiently cluster dynamically changing data through split and merge processes at each time step, where the decision for splitting and merging is made by approximate posterior distributions over partitions with Dirichlet Process (DP) priors. Moreover, BSMC sidesteps the difficult problem of finding the proper number of clusters by virtue of the flexibility of nonparametric Bayesian models. We naturally apply BSMC to online video segmentation, which is composed of three steps-pixel clustering, histogram-based merging and temporal matching. We demonstrate the performance of our algorithm on complex real video sequences compared to other existing methods.

Minimum entropy, k-means, spectral clustering

2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541)

Bayesian Hierarchical Clustering with Exponential Family: Small-Variance Asymptotics and Reducibility

arXiv (Cornell University), Jan 29, 2015

Bayesian hierarchical clustering (BHC) is an agglomerative clustering method, where a probabilist... more Bayesian hierarchical clustering (BHC) is an agglomerative clustering method, where a probabilistic model is defined and its marginal likelihoods are evaluated to decide which clusters to merge. While BHC provides a few advantages over traditional distance-based agglomerative clustering algorithms, successive evaluation of marginal likelihoods and careful hyperparameter tuning are cumbersome and limit the scalability. In this paper we relax BHC into a non-probabilistic formulation, exploring smallvariance asymptotics in conjugate-exponential models. We develop a novel clustering algorithm, referred to as relaxed BHC (RBHC), from the asymptotic limit of the BHC model that exhibits the scalability of distance-based agglomerative clustering algorithms as well as the flexibility of Bayesian nonparametric models. We also investigate the reducibility of the dissimilarity measure emerged from the asymptotic limit of the BHC model, allowing us to use scalable algorithms such as the nearest neighbor chain algorithm. Numerical experiments on both synthetic and real-world datasets demonstrate the validity and high performance of our method.

On Uncertainty Estimation by Tree-based Surrogate Models in Sequential Model-based Optimization

arXiv (Cornell University), Feb 21, 2022

Sequential model-based optimization sequentially selects a candidate point by constructing a surr... more Sequential model-based optimization sequentially selects a candidate point by constructing a surrogate model with the history of evaluations, to solve a black-box optimization problem. Gaussian process (GP) regression is a popular choice as a surrogate model, because of its capability of calculating prediction uncertainty analytically. On the other hand, an ensemble of randomized trees is another option and has practical merits over GPs due to its scalability and easiness of handling continuous/discrete mixed variables. In this paper we revisit various ensembles of randomized trees to investigate their behavior in the perspective of prediction uncertainty estimation. Then, we propose a new way of constructing an ensemble of randomized trees, referred to as BwO forest, where bagging with oversampling is employed to construct bootstrapped samples that are used to build randomized trees with random splitting. Experimental results demonstrate the validity and good performance of BwO forest over existing tree-based models in various circumstances.

Blind separation of nonstationary and temporally correlated sources from noisy mixtures

Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501)

Local Stability Analysis Of Flexible Independent Component Analysis Algorithm

Acoustics, Speech, and Signal Processing, 1988. ICASSP-88., 1988 International Conference on

This paper addresses local stability analysis for the exible independent component analysis (ICA)... more This paper addresses local stability analysis for the exible independent component analysis (ICA) algorithm 6] where the generalized Gaussian density m o d e l w as employed for blind separation of mixtures of sub-and super-Gaussian sources. In the exible ICA algorithm, the shape of nonlinear function in the learning algorithm varies depending on the Gaussian exponent which is properly selected according to the kurtosis of estimated source. In the framework of the natural gradient in Stiefel manifold, the exible ICA algorithm is revisited and some new results about its local stability analysis are presented.

Topographic Independent Component Analysis of Gene Expression Time Series Data

Topographic independent component analysis (TICA) is an interesting extension of the conventional... more Topographic independent component analysis (TICA) is an interesting extension of the conventional ICA, which aims at finding a linear decomposition into approximately independent components with the dependence between two components is approximated by their proximity in the topographic representation. In this paper we apply the topographic ICA to gene expression time series data and compare it with the conventional ICA as well as the independent subspace analysis (ISA). Empirical study with yeast cell cycle-related data and yeast sporulation data, shows that TICA is more suitable for gene clustering.

Iterative Projection Approximation Algorithms for PCA

2006 IEEE International Conference on Acoustics Speed and Signal Processing Proceedings

In this paper we introduce a new error measure, integrated reconstruction error (IRE), the minimi... more In this paper we introduce a new error measure, integrated reconstruction error (IRE), the minimization of which leads to principal eigenvectors (without rotational ambiguity) of the data covariance matrix. Then we present iterative algorithms for the IRE minimization, through the projection approximation. The proposed algorithm is referred to as COnstrained Projection Approximation (COPA) algorithm and its limiting case is called COPAL. We also discuss regularized algorithms, referred to as R-COPA and R-COPAL. Numerical experiments demonstrate that these algorithms successfully find exact principal eigenvectors of the data covariance matrix.

Bayesian Multi-task Learning for Common Spatial Patterns

2011 International Workshop on Pattern Recognition in NeuroImaging, 2011

Common spatial pattern (CSP) is a widely-used feature extraction method for electroencephalogram ... more Common spatial pattern (CSP) is a widely-used feature extraction method for electroencephalogram (EEG) classification and corresponding probabilistic models were recently developed, adopting a linear generative model for each class. These models are trained on a subject-by-subject basis so that inter-subject information is neglected. Moreover when only a few training samples are available for each subject, the performance is degraded. In this paper we employ Bayesian multi-task learning so that subject-to-subject information is transferred in learning the model for a subject of interest. We present two probabilistic models where precision parameters of multivariate or matrix-variate Gaussian prior for the dictionary are shared across subjects. Numerical experiments on the BCI competition IV 2a dataset confirm that our methods improve classification performance over the standard CSP (on a subject-by-subject basis), especially in the case of subjects with fewer number of training samples.

Sequential Spectral Learning to Hash with Multiple Representations

Computer Vision – ECCV 2012, 2012

Learning to hash involves learning hash functions from a set of images for embedding high-dimensi... more Learning to hash involves learning hash functions from a set of images for embedding high-dimensional visual descriptors into a similarity-preserving low-dimensional Hamming space. Most of existing methods resort to a single representation of images, that is, only one type of visual descriptors is used to learn a hash function to assign binary codes to images. However, images are often described by multiple different visual descriptors (such as SIFT, GIST, HOG), so it is desirable to incorporate these multiple representations into learning a hash function, leading to multi-view hashing. In this paper we present a sequential spectral learning approach to multi-view hashing where a hash function is sequentially determined by solving the successive maximization of local variances subject to decorrelation constraints. We compute multi-view local variances by α-averaging view-specific distance matrices such that the best averaged distance matrix is determined by minimizing its α-divergence from view-specific distance matrices. We also present a scalable implementation, exploiting a fast approximate k-NN graph construction method, in which α-averaged distances computed in small partitions determined by recursive spectral bisection are gradually merged in conquer steps until whole examples are used. Numerical experiments on Caltech-256, CIFAR-20, and NUS-WIDE datasets confirm the high performance of our method, in comparison to single-view spectral hashing as well as existing multi-view hashing methods.

Hashing with Generalized Nyström Approximation

2012 IEEE 12th International Conference on Data Mining, 2012

Hashing, which involves learning binary codes to embed high-dimensional data into a similarity-pr... more Hashing, which involves learning binary codes to embed high-dimensional data into a similarity-preserving low-dimensional Hamming space, is often formulated as linear dimensionality reduction followed by binary quantization. Linear dimensionality reduction, based on maximum variance formulation, requires leading eigenvectors of data covariance or graph Laplacian matrix. Computing leading singular vectors or eigenvectors in the case of high-dimension and large sample size, is a main bottleneck in most of data-driven hashing methods. In this paper we address the use of generalized Nyström method where a subset of rows and columns are used to approximately compute leading singular vectors of the data matrix, in order to improve the scalability of hashing methods in the case of high-dimensional data with large sample size. Especially we validate the useful behavior of generalized Nyström approximation with uniform sampling, in the case of a recentlydeveloped hashing method based on principal component analysis (PCA) followed by an iterative quantization, referred to as PCA+ITQ, developed by Gong and Lazebnik. We compare the performance of generalized Nyström approximation with uniform and non-uniform sampling, to the full singular value decomposition (SVD) method, confirming that the uniform sampling improves the computational and space complexities dramatically, while the performance is not much sacrificed. In addition we present low-rank approximation error bounds for generalized Nyström approximation with uniform sampling, which is not a trivial extension of available results on the nonuniform sampling case.

Multi-view anchor graph hashing

2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 2013

Multi-view hashing seeks compact integrated binary codes which preserve similarities averaged ove... more Multi-view hashing seeks compact integrated binary codes which preserve similarities averaged over multiple representations of objects. Most of existing multi-view hashing methods resort to linear hash functions where data manifold is not considered. In this paper we present multi-view anchor graph hashing (MVAGH), where nonlinear integrated binary codes are efficiently determined by a subset of eigenvectors of an averaged similarity matrix. The efficiency behind MVAGH is due to a low-rank form of the averaged similarity matrix induced by multi-view anchor graph, where the similarity between two points is measured by two-step transition probability through view-specific anchor (i.e. landmark) points. In addition, we observe that MVAGH suffers from the performance degradation when the high recall is required. To overcome this drawback, we propose a simple heuristic to combine MVAGH with locality sensitive hashing (LSH). Numerical experiments on CIFAR-10 dataset confirms that MVAGH(+LSH) outperforms the existing multi-and single-view hashing methods.

Incremental Tree-Based Inference with Dependent Normalized Random Measures

Normalized random measures (NRMs) form a broad class of discrete random measures that are used as... more Normalized random measures (NRMs) form a broad class of discrete random measures that are used as priors for Bayesian nonparametric models. Dependent normalized random measures (DNRMs) introduce dependencies in a set of NRMs, to facilitate the handling of data where the assumption of exchangeability is violated. Various methods have been developed to construct DNRMs; of particular interest is mixed normalized random measures (MNRMs), where DNRM is represented as a mixture of underlying shared normalized random measures. Emphasis in existing works is placed on the construction methods of DNRMs, but there is a little work on efficient inference for DNRMs. In this paper, we present a tree-based inference method for MNRM mixture models, extending Bayesian hierarchical clustering (BHC) which was originally developed as a deterministic approximate inference for Dirichlet process mixture (DPM) models. We also present an incremental inference for MNRM mixture models, building a tree incrementally in the sense that the tree structure is partially updated whenever a new data point comes in. The tree, when constructed in such a way, allows us to efficiently perform tree-consistent MAP inference in MRNM mixture models, determining a most probable tree-consistent partition, as well as to compute a marginal likelihood approximately. Numerical experiments on both synthetic and real-world datasets demonstrate the usefulness of our algorithm, compared to MCMC methods.

Clustering sequence sets for motif discovery

Most of existing methods for DNA motif discovery consider only a single set of sequences to find ... more Most of existing methods for DNA motif discovery consider only a single set of sequences to find an over-represented motif. In contrast, we consider multiple sets of sequences where we group sets associated with the same motif into a cluster, assuming that each set involves a single motif. Clustering sets of sequences yields clusters of coherent motifs, improving signal-to-noise ratio or enabling us to identify multiple motifs. We present a probabilistic model for DNA motif discovery where we identify multiple motifs through searching for patterns which are shared across multiple sets of sequences. Our model infers cluster-indicating latent variables and learns motifs simultaneously, where these two tasks interact with each other. We show that our model can handle various motif discovery problems, depending on how to construct multiple sets of sequences. Experiments on three different problems for discovering DNA motifs emphasize the useful behavior and confirm the substantial gains over existing methods where only a single set of sequences is considered.

Self-labeling for P300 detection

2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2012

The P300 wave refers to a positive peak with a latency of 300 ms, produced in response to task-re... more The P300 wave refers to a positive peak with a latency of 300 ms, produced in response to task-relevant stimuli. It is a widely-used event related potential (ERP) in practical brain computer interface (BCI) systems, where a classifier is trained using discriminative features extracted from a set of labeled examples, in order to detect the presence of P300. Given a small training examples, the intra-and inter-subject variations in amplitude and latency of P300 degrade the performance of classifier. Thus, it requires a longer training period (calibration time) with more labeled examples for satisfactory performance. In this paper we present a self-labeling method, where confident unlabeled data, together with their predicted labels, are gradually added to the training set, in order to re-train the classifier. Linear discriminant analysis with singular value decomposition is used to progressively extract discriminate features. Experiments demonstrate the high performance of our method, especially in the case where a small number of training examples are available. We also apply the method to the zero-calibration P300-based BCI, which removes subject-dependent calibration procedures by using the training set already recorded from other subjects.

Learning α-integration with partially-labeled data

2010 IEEE International Conference on Acoustics, Speech and Signal Processing, 2010

Sensory data integration is an important task in human brain for multimodal processing as well as... more Sensory data integration is an important task in human brain for multimodal processing as well as in machine learning for multisensor processing. α-integration was proposed by Amari as a principled way of blending multiple positive measures (e.g., stochastic models in the form of probability distributions), providing an optimal integration in the sense of minimizing the α-divergence. It also encompasses existing integration methods as its special case, e.g., weighted average and exponential mixture. In α-integration, the value of α determines the characteristics of the integration and the weight vector w assigns the degree of importance to each measure. In most of the existing work, however, α and w are given in advance rather than learned. In this paper we present two algorithms, for learning α and w from data when only a few integrated target values are available. Numerical experiments on synthetic as well as real-world data confirm the proposed method's effectiveness.

Source Separation with Gaussian Process Models

Lecture Notes in Computer Science

Online multi-label learning with accelerated nonsmooth stochastic gradient descent

2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 2013

Rao-Blackwellized Particle Filtering for Sequential Speech Enhancement

The 2006 IEEE International Joint Conference on Neural Network Proceedings

In this paper we present a method of sequential speech enhancement, where we infer clean speech s... more In this paper we present a method of sequential speech enhancement, where we infer clean speech signal using a Rao-Blackwellized particle filter (RBPF), given a noisecontaminated observed signal. In contrast to Kalman filteringbased methods, we consider a non-Gaussian speech generative model that is based on the generalized auto-regressive (GAR) model. Model parameters are learned by sequential expectation maximization, incorporating the RBPF. Empirical comparison to Kalman filter, confirms the high performance of the proposed method.

Orthogonal Nonnegative Matrix Factorization for Blind Image Separation

Lecture Notes in Computer Science, 2013

Online Video Segmentation by Bayesian Split-Merge Clustering

Lecture Notes in Computer Science, 2012

We present an online video segmentation algorithm based on a novel nonparametric Bayesian cluster... more We present an online video segmentation algorithm based on a novel nonparametric Bayesian clustering method called Bayesian Split-Merge Clustering (BSMC). BSMC can efficiently cluster dynamically changing data through split and merge processes at each time step, where the decision for splitting and merging is made by approximate posterior distributions over partitions with Dirichlet Process (DP) priors. Moreover, BSMC sidesteps the difficult problem of finding the proper number of clusters by virtue of the flexibility of nonparametric Bayesian models. We naturally apply BSMC to online video segmentation, which is composed of three steps-pixel clustering, histogram-based merging and temporal matching. We demonstrate the performance of our algorithm on complex real video sequences compared to other existing methods.

Minimum entropy, k-means, spectral clustering

2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541)