Machine-learned cluster identification in high-dimensional data

IEEE Access

Clustering is a challenging problem in machine learning in which one attempts to group N objects into K 0 groups based on P features measured on each object. In this article, we examine the case where N P and K 0 is not known. Clustering in such high dimensional, small sample size settings has numerous applications in biology, medicine, the social sciences, clinical trials, and other scientific and experimental fields. Whereas most existing clustering algorithms either require the number of clusters to be known a priori or are sensitive to the choice of tuning parameters, our method does not require the prior specification of K 0 or any tuning parameters. This represents an important advantage for our method because training data are not available in the applications we consider (i.e., in unsupervised learning problems). Without training data, estimating K 0 and other hyperparameters-and thus applying alternative clustering algorithms-can be difficult and lead to inaccurate results. Our method is based on a simple transformation of the Gram matrix and application of the strong law of large numbers to the transformed matrix. If the correlation between features decays as the number of features grows, we show that the transformed feature vectors concentrate tightly around their respective cluster expectations in a low-dimensional space. This result simplifies the detection and visualization of the unknown cluster configuration. We illustrate the algorithm by applying it to 32 benchmarked microarray datasets, each containing thousands of genomic features measured on a relatively small number of tissue samples. Compared to 21 other commonly used clustering methods, we find that the proposed algorithm is faster and twice as accurate in determining the ''best'' cluster configuration. INDEX TERMS Clustering, gram matrix, high-dimensional features, hyperparameter-free.

2005

In this paper, we propose a new clustering method consisting in automated “flood- fill segmentation” of the U*-matrix of a Self-Organizing Map after training. Using several artificial datasets as a benchmark, we find that the clustering results of our U*F method are good over a wide range of critical dataset types. Furthermore, comparison to standard clustering algorithms (K-means, single-linkage and Ward) directly applied on the same datasets show that each of the latter performs very bad on at least one kind of dataset, contrary to our U*F clustering method: while not always the best, U*F clustering has the great advantage of exhibiting consistently good results. Another advantage of U*F is that the computation cost of the SOM segmentation phase is negligible, contrary to other SOM-based clustering approaches which apply O(n2logn) standard clustering algorithms to the SOM prototypes. Finally, it should be emphasized that U*F clustering does not require a priori knowledge on the nu...

2011

Abstract: We survey agglomerative hierarchical clustering algorithms and discuss efficient implementations that are available in R and other software environments. We look at hierarchical self-organizing maps, and mixture models. We review grid-based clustering, focusing on hierarchical density-based approaches. Finally we describe a recently developed very efficient (linear time) hierarchical clustering algorithm, which can also be viewed as a hierarchical grid-based algorithm.

Clustering is a branch of multivariate analysis that is used to create groups of data. While there are currently a variety of techniques that are used for creating clusters, many require defining additional information, including the actual number of clusters, before they can be carried out. The case study of this research presents a novel neural network that is capable of creating groups by using a combination of hierarchical clustering and self-organizing maps, without requiring the number of existing clusters to be specified beforehand.

2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541)

The Self-Organizing Map (SOM) has emerged as one of the popular choices for clustering data; however, when it comes to point density accuracy of codebooks or reliability and interpretability of the map, the SOM leaves much to be desired. In this paper, we compare the newly developed K-Means Hierarchical (KMH) clustering algorithm to the SOM. We also introduce a new initialization scheme for the K-means that improves codebook placement and, propose a novel visualization scheme that combines the Principal Component Analysis (PCA) and Minimal Spanning Tree (MST) in an arrangement that ensures reliability of the visualization unlike the SOM. A practical application of the algorithm is demonstrated on a challenging Bioinformatics problem.

PLoS ONE, 2008

Background: A hierarchy, characterized by tree-like relationships, is a natural method of organizing data in various domains. When considering an unsupervised machine learning routine, such as clustering, a bottom-up hierarchical (BU, agglomerative) algorithm is used as a default and is often the only method applied.

Studies in health technology and informatics, 2002

This work deals with multidimensional data analysis, precisely cluster analysis applied to a very well known dataset, the Wisconsin Breast Cancer dataset. After the introduction of the topics of the paper the cluster analysis concept is shortly explained and different methods of cluster analysis are compared. Further, the Kohonen model of self-organizing maps is briefly described together with an example and with explanations of how the cluster analysis can be performed using the maps. After describing the data set and the methodology used for the analysis we present the findings using textual as well as visual descriptions and conclude that the approach is a useful complement for assessing multidimensional data and that this dataset has been overused for automated decision benchmarking purposes, without a thorough analysis of the data it contains.

Self-Organizing Maps, 2010

Applications of clustering algorithms in biomedical research are ubiquitous, with typical examples including gene expression data analysis, genomic sequence analysis, biomedical document mining, and MRI image analysis. However, due to the diversity of cluster analysis, the differing terminologies, goals, and assumptions underlying different clustering algorithms can be daunting. Thus, determining the right match between clustering algorithms and biomedical applications has become particularly important. This paper is presented to provide biomedical researchers with an overview of the status quo of clustering algorithms, to illustrate examples of biomedical applications based on cluster analysis, and to help biomedical researchers select the most suitable clustering algorithms for their own applications. Fig. 1. Number of papers on cluster analysis in subject areas of Life and Health Sciences from 2000 to 2009. Searches were performed using Scopus. in the GenBank database, constituting an increase of about 25 billion bases from two years ago [89,172,350,468 bases from 85,500,730 sequences, released on April 15, 2008 (GenBank Release 165.0)].

Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2011

We survey agglomerative hierarchical clustering algorithms and discuss efficient implementations that are available in R and other software environments. We look at hierarchical self-organizing maps, and mixture models. We review grid-based clustering, focusing on hierarchical density-based approaches. Finally, we describe a recently developed very efficient (linear time) hierarchical clustering algorithm, which can also be viewed as a hierarchical grid-based algorithm. C

Lecture Notes in Computer Science, 2016

One main challenge in modern medicine is the discovery of molecular disease subtypes characterized by relevant clinical differences, such as survival. However, clustering high-dimensional expression data is challenging due to noise and the curse of high-dimensionality. This article describes a disease subtyping pipeline that is able to exploit the important information available in pathway databases and clinical variables. The pipeline consists of a new feature selection procedure and existing clustering methods. Our procedure partitions a set of patients using the set of genes in each pathway as clustering features. To select the best features, this procedure estimates the relevance of each pathway and fuses relevant pathways. We show that our pipeline finds subtypes of patients with more distinctive survival profiles than traditional subtyping methods by analyzing a TCGA colon cancer gene expression dataset. Here we demonstrate that our pipeline improves three different clustering methods: k-means, SNF, and hierarchical clustering.

International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 2021

Artificial Intelligence (AI) and Machine Learning (ML), which are becoming a part of interest rapidly for various researchers. ML is the field of Computer Science study, which gives capability to learn without being absolutely programmed. This work focuses on the standard k-means clustering algorithm and analysis the shortcomings of the standard k-means algorithm. The k-means clustering algorithm calculates the distance between each data object and not all cluster centres in every iteration, which makes the efficiency of clustering is high. In this work, we have to try to improve the k-means algorithm to solve simple data to store some information in every iteration, which is to be used in the next interaction. This method avoids computing distance of data object to the cluster centre repeatedly, saving the running time. An experimental result shows the enhanced speed of clustering, accuracy, reducing the computational complexity of the k-means. In this, we have work on iris dataset extracted from Kaggle.

2004

The development of microarray technologies gives scientists the ability to examine, discover and monitor the mRNA transcript levels of thousands of genes in a single experiment. Nevertheless, the tremendous amount of data that can be obtained from microarray studies presents a challenge for data analysis. The most commonly used computational approach for analyzing microarray data is cluster analysis. In this paper, we investigate the application of an unsupervised extension of the recently proposed k-windows clustering algorithm on gene expression microarray data. This algorithm apart from identifying the clusters present in a dataset also calculates their number thus no special knowledge about the data is required. To improve the quality of the clustering, we selected the most highly correlated genes with respect to the class distinction of the genes. The results obtained by the application of the algorithm exhibit high classification success.

Journal of Neuroscience Methods, 2005

Cluster analysis is an important tool for classifying data. Established techniques include k-means and k-median cluster analysis. However, these methods require the user to provide a priori estimations of the number of clusters and their approximate location in the parameter space. Often these estimations can be made based on some prior understanding about the nature of the data. Alternatively, the user makes these estimations based on visualization of the data. However, the latter is problematic in data sets with large numbers of dimensions. Presented here is an algorithm that can automatically provide these estimates without human intervention based on the inherent structure of the data set. The number of dimensions does not limit it.

2004

In this paper, we investigate the application of an unsupervised extension of the recently proposed k-windows clustering algorithm on gene expression microarray data. The k-windows algorithm is used both to identify sets of genes according to their expression in a set of samples, and to cluster samples into homogeneous groups. Experimental results and comparisons indicate that this is a promising approach.