Skip to main content

Glenn Milligan

Followers

140

Following

2

Co-authors

4

Public Views

Montana State University - Bozeman

Vania V Estrela

UFF - Universidade Federal Fluminense

Adebayo D Agunbiade

Olabisi Onabanjo University, Ago-Iwoye, Nigeria

New Mexico State University

Srinivasa Rao Gadde

The University of Dodoma, Tanzania

Nikos Mantzakouras

National & Kapodistrian University of Athens

Yildiz Technical University

Armando Marques-Guedes

UNL - New University of Lisbon

Gwen Robbins Schug

University of North Carolina at Greensboro

Gabriel Gutierrez-Alonso

University of Salamanca

Interests

Uploads

Papers by Glenn Milligan

Asymptotic and Finite Sample Characteristics of Four External Criterion Measures

by Glenn Milligan and David Schilling

Multivariate Behav Res, 1985

ABSTRACT Finite and asymptotic formulas are derived for the Rand, adjusted Rand, Fowlkes and Mall... more ABSTRACT Finite and asymptotic formulas are derived for the Rand, adjusted Rand, Fowlkes and Mallows, and Jaccard external criterion measures. The formulas were developed for ten clustering situations which have been used previously in the clustering literature. The results indicated that the Jaccard and Fowlkes and Mallows measures were fairly similar. The Rand and adjusted Rand indices formed another similar pair. Weaknesses involving the Rand and the Fowlkes and Mallows indices are noted. Based on these and previous simulation results, it is recommended that the adjusted Rand and Jaccard measures be used in validation studies of clustering algorithms.

An examination of procedures for determining the number of clusters in a data set

Psychometrika, Jun 1, 1985

... It would be hard to believe that a method that fails on the present data would be able to per... more

A comment on the use of canonical correlation in the analysis of contingency tables

Psychol Bull, 1983

Holland, Levi, and Watson (1980) proposed the use of canonical correlation for the analysis of co... more

The validation of four ultrametric clustering algorithms

Pattern Recognition, 1980

The present simulation study examined the ability of four hierarchical clustering algorithms to r... more The present simulation study examined the ability of four hierarchical clustering algorithms to recover the true structure in data sets which satisfied both the ultrametric inequality and the structural model of the clustering procedures. The results indicated that the rank order performance of the four methods differed markedly from the rank order generally found in multivariate normal mixture model studies. The differing rank orders demonstrates a lack of robustness of the algorithms over alternative conceptualizations of cluster structure.

Clustering and Classification Methods

by Glenn Milligan and Stephen Hirtle

Handbook of Psychology, 2003

The purpose of this chapter is to provide a review of the current state of knowledge in the field... more The purpose of this chapter is to provide a review of the current state of knowledge in the field of clustering and classification as applied in the behavioral sciences. Because of the extensive literature base and the wide range of application areas, no attempt or assurance can be made that all domains of study in this area have been covered. Rather, the main research themes and well-known algorithms are reviewed. In addition, the chapter includes a survey of the issues critical to the analysis of empirical data with recommendations for ...

An examination of procedures for determining the number of clusters in a data set

Psychometrika, 1985

... It would be hard to believe that a method that fails on the present data would be able to per... more

Robustness properties of nonorthogonal analysis of variance

Psychological Bulletin, 1987

Page 1. QUANTITATIVE METHODS IN PSYCHOLOGY Robustness Properties of Nonorthogonal Analysis of Var... more

Mapping Influence Regions in Heirarchical Clustering

Multivariate Behavioral Research, 1995

Three-dimensional response surface plots arc presented for several hierarchical clustering method... more

Asymptotic and Finite Sample Characteristics of Four External Criterion Measures

Multivariate Behavioral Research, 1985

ABSTRACT Finite and asymptotic formulas are derived for the Rand, adjusted Rand, Fowlkes and Mall... more ABSTRACT Finite and asymptotic formulas are derived for the Rand, adjusted Rand, Fowlkes and Mallows, and Jaccard external criterion measures. The formulas were developed for ten clustering situations which have been used previously in the clustering literature. The results indicated that the Jaccard and Fowlkes and Mallows measures were fairly similar. The Rand and adjusted Rand indices formed another similar pair. Weaknesses involving the Rand and the Fowlkes and Mallows indices are noted. Based on these and previous simulation results, it is recommended that the adjusted Rand and Jaccard measures be used in validation studies of clustering algorithms.

A Study of the Comparability of External Criteria for Hierarchical Cluster Analysis

Multivariate Behavioral Research, 1986

... OCTOBER 1986 453 Page 14. Glenn W. Milligan and Martha C. Cooper ... Values greater than abou... more

Approaches to mass customization: configurations and empirical validation

by Glenn Milligan and Rebecca Duray

Journal of Operations Management, 2000

Page 1. . Journal of Operations Management 18 2000 605625 www.elsevier. comrlocaterdsw Approac... more

A study of standardization of variables in cluster analysis

Journal of Classification, 1988

A methodological problem in applied clustering involves the decision of whether or not to standar... more A methodological problem in applied clustering involves the decision of whether or not to standardize the input variables prior to the computation of a Euclidean distance dissimilarity measure. Existing results have been mixed with some studies recommending standardization and others suggesting that it may not be desirable. The existence of numerous approaches to standardization complicates the decision process. The present simulation study examined the standardization problem. A variety of data structures were generated which varied the intercluster spacing and the scales for the variables. The data sets were examined in four different types of error environments. These involved error free data, error perturbed distances, inclusion of outliers, and the addition of random noise dimensions. Recovery of true cluster structure as found by four clustering methods was measured at the correct partition level and at reduced levels of coverage. Results for eight standardization strategies are presented. It was found that those approaches which standardize by division by the range of the variable gave consistently superior recovery of the underlying cluster structure. The result held over different error conditions, separation distances, clustering methods, and coverage levels. The traditional z-score transformation was found to be less effective in several situations.

Measuring the influence of individual data points in a cluster analysis

Journal of Classification, 1996

The problem of measuring the impact of individual data points in a cluster analysis is examined. ... more The problem of measuring the impact of individual data points in a cluster analysis is examined. The purpose is to identify those data points that have an influence on the resulting cluster partitions. Influence of a single data point is considered present when different cluster partitions result from the removal of the element from the data set. The Hubert and Arable corrected Rand index was used to provide numerical measures of influence of a data point. Simulated data sets consisting of a variety of cluster structures and error conditions were generated to validate the influence measures. The results showed that the measure of internal influence was 100% accurate in identifying those data elements exhibiting an influential effect. The nature of the influence, whether beneficial or detrimental to the clustering, can be evaluated with the use of the gamma and point-biserial statistics.

The Effect of Cluster Size, Dimensionality, and the Number of Clusters on Recovery of True Cluster Structure

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000

An evaluation of four clustering methods and four extemal criterion measures was conducted with r... more An evaluation of four clustering methods and four extemal criterion measures was conducted with respect to the effect of the number of clusters, dimensionality, and relative cluster sizes on the recovery of true cluster structure. The four methods were the single link, complete link, group average (UPGMA), and Ward's minimum variance algorithms. The results indicated that the four criterion measures were generally consistent with each other, of which two highly similar pairs were identified. The tirst pair consisted of the Rand and corrected Rand statistics, and the second pair was the Jaccard and the Fowlkes and Mallows indexes. With respect to the methods, recovery was found to improve as the number of clusters increased and as the number of dimensions increased. The relative cluster size factor produced differential perfonnance effects, with Ward's procedure providing the best recovery when the clusters were of equal size. The group average method gave equivalent or better recovery when the clusters were of unequal size.

Hierarchical Clustering Algorithms with Influence Detection

Educational and Psychological Measurement, 1995

Abstract The computer program described in this study is based on the methodology developed by th... more

K-Means Clustering Methods with Influence Detection

Educational and Psychological Measurement, 1996

Abstract A software program is described that performs K-means clustering with several enhancemen... more

A Two-Stage Clustering Algorithm with Robust Recovery Characteristics

Educational and Psychological Measurement, 1980

Two FORTRAN IV computer programs for a two-stage clustering algorithm with robust recovery charac... more Two FORTRAN IV computer programs for a two-stage clustering algorithm with robust recovery characteristics are described. In Stage 1, a group average hierarchical algorithm generates cluster centroids which are used as starting seed points for Stage 2, Jancey&amp;amp;amp;amp;#39;s k-means nonhierarchical algorithm. Also available with the hierarchical algorithm is a hypothesis test procedure which can be used to determine whether significant

An Algorithm for Calculating Coefficients Required for Trend Analysis

Educational and Psychological Measurement, 1980

A NOTE ON PROCEDURES FOR TESTING THE QUALITY OF A CLUSTERING OF A SET OF OBJECTS

Decision Sciences, 1980

Despite the increased application of cluster analysis in decision sciences, few attempts have bee... more Despite the increased application of cluster analysis in decision sciences, few attempts have been made to derive hypothesis-testing procedures for the evaluation of clustering solutions. In fact, the present paper shows that at least one such attempt failed to specify a meaningful sampling distribution for the test procedure. An alternative index based on the concept of point-biserial correlation is proposed as a possible recovery measure. The index is subsequently used to form the basis of a valid statistical test for the existence of cluster structure.

$Input Matrix of Direct Dissimilarities ——eo ws Oe eeeeeoerlcrOoOor rehome eee NOISES SEES “eae In order to test the null hypothesis of no cluster structure at each level in the hierarchy, 100 random-noise data sets were generated. A multivariate uniform distribution was used to generate the data. All variables were uncorrelated in the distribution. The random-noise sets contained the same number of elements as in the data set of interest. Clustering solutions for these data sets were obtained, and point-biserial correlations were computed for all cluster levels. Columns 4 and 5 in Table 3 give the mean and the standard deviation of the null distribution for each hierarchy level. Finally, the third column of Table 3 gives the level of signifi- cance. That is, the significance level indicates the fraction of the 100 random- noise data sets with values for the point-biserial correlation which exceeded that of the empirica] data set being analyzed.$

Methodology Review: Clustering Methods

Applied Psychological Measurement, 1987

A review of clustering methodology is presented, with emphasis on algorithm performance and the r... more A review of clustering methodology is presented, with emphasis on algorithm performance and the resulting implications for applied research. After an overview of the clustering literature, the clustering process is discussed within a seven-step framework. The four major types of clustering methods can be characterized as hierarchical, partitioning, overlapping, and ordination algorithms. The validation of such algorithms refers to the problem of determining the ability of the methods to recover cluster configurations which are known to exist in the data. Validation approaches include mathematical derivations, analyses of empirical datasets, and monte carlo simulation methods. Next, interpretation and inference procedures in cluster analysis are discussed. inference procedures involve testing for significant cluster structure and the problem of determining the number of clusters in the data. The paper concludes with two sets of recommendations.

Asymptotic and Finite Sample Characteristics of Four External Criterion Measures

by Glenn Milligan and David Schilling

Multivariate Behav Res, 1985

ABSTRACT Finite and asymptotic formulas are derived for the Rand, adjusted Rand, Fowlkes and Mall... more ABSTRACT Finite and asymptotic formulas are derived for the Rand, adjusted Rand, Fowlkes and Mallows, and Jaccard external criterion measures. The formulas were developed for ten clustering situations which have been used previously in the clustering literature. The results indicated that the Jaccard and Fowlkes and Mallows measures were fairly similar. The Rand and adjusted Rand indices formed another similar pair. Weaknesses involving the Rand and the Fowlkes and Mallows indices are noted. Based on these and previous simulation results, it is recommended that the adjusted Rand and Jaccard measures be used in validation studies of clustering algorithms.

An examination of procedures for determining the number of clusters in a data set

Psychometrika, Jun 1, 1985

... It would be hard to believe that a method that fails on the present data would be able to per... more

A comment on the use of canonical correlation in the analysis of contingency tables

Psychol Bull, 1983

Holland, Levi, and Watson (1980) proposed the use of canonical correlation for the analysis of co... more

The validation of four ultrametric clustering algorithms

Pattern Recognition, 1980

The present simulation study examined the ability of four hierarchical clustering algorithms to r... more The present simulation study examined the ability of four hierarchical clustering algorithms to recover the true structure in data sets which satisfied both the ultrametric inequality and the structural model of the clustering procedures. The results indicated that the rank order performance of the four methods differed markedly from the rank order generally found in multivariate normal mixture model studies. The differing rank orders demonstrates a lack of robustness of the algorithms over alternative conceptualizations of cluster structure.

Clustering and Classification Methods

by Glenn Milligan and Stephen Hirtle

Handbook of Psychology, 2003

The purpose of this chapter is to provide a review of the current state of knowledge in the field... more The purpose of this chapter is to provide a review of the current state of knowledge in the field of clustering and classification as applied in the behavioral sciences. Because of the extensive literature base and the wide range of application areas, no attempt or assurance can be made that all domains of study in this area have been covered. Rather, the main research themes and well-known algorithms are reviewed. In addition, the chapter includes a survey of the issues critical to the analysis of empirical data with recommendations for ...

An examination of procedures for determining the number of clusters in a data set

Psychometrika, 1985

... It would be hard to believe that a method that fails on the present data would be able to per... more

Robustness properties of nonorthogonal analysis of variance

Psychological Bulletin, 1987

Page 1. QUANTITATIVE METHODS IN PSYCHOLOGY Robustness Properties of Nonorthogonal Analysis of Var... more

Mapping Influence Regions in Heirarchical Clustering

Multivariate Behavioral Research, 1995

Three-dimensional response surface plots arc presented for several hierarchical clustering method... more

Asymptotic and Finite Sample Characteristics of Four External Criterion Measures

Multivariate Behavioral Research, 1985

ABSTRACT Finite and asymptotic formulas are derived for the Rand, adjusted Rand, Fowlkes and Mall... more ABSTRACT Finite and asymptotic formulas are derived for the Rand, adjusted Rand, Fowlkes and Mallows, and Jaccard external criterion measures. The formulas were developed for ten clustering situations which have been used previously in the clustering literature. The results indicated that the Jaccard and Fowlkes and Mallows measures were fairly similar. The Rand and adjusted Rand indices formed another similar pair. Weaknesses involving the Rand and the Fowlkes and Mallows indices are noted. Based on these and previous simulation results, it is recommended that the adjusted Rand and Jaccard measures be used in validation studies of clustering algorithms.

A Study of the Comparability of External Criteria for Hierarchical Cluster Analysis

Multivariate Behavioral Research, 1986

... OCTOBER 1986 453 Page 14. Glenn W. Milligan and Martha C. Cooper ... Values greater than abou... more

Approaches to mass customization: configurations and empirical validation

by Glenn Milligan and Rebecca Duray

Journal of Operations Management, 2000

Page 1. . Journal of Operations Management 18 2000 605625 www.elsevier. comrlocaterdsw Approac... more

A study of standardization of variables in cluster analysis

Journal of Classification, 1988

A methodological problem in applied clustering involves the decision of whether or not to standar... more A methodological problem in applied clustering involves the decision of whether or not to standardize the input variables prior to the computation of a Euclidean distance dissimilarity measure. Existing results have been mixed with some studies recommending standardization and others suggesting that it may not be desirable. The existence of numerous approaches to standardization complicates the decision process. The present simulation study examined the standardization problem. A variety of data structures were generated which varied the intercluster spacing and the scales for the variables. The data sets were examined in four different types of error environments. These involved error free data, error perturbed distances, inclusion of outliers, and the addition of random noise dimensions. Recovery of true cluster structure as found by four clustering methods was measured at the correct partition level and at reduced levels of coverage. Results for eight standardization strategies are presented. It was found that those approaches which standardize by division by the range of the variable gave consistently superior recovery of the underlying cluster structure. The result held over different error conditions, separation distances, clustering methods, and coverage levels. The traditional z-score transformation was found to be less effective in several situations.

Measuring the influence of individual data points in a cluster analysis

Journal of Classification, 1996

The problem of measuring the impact of individual data points in a cluster analysis is examined. ... more The problem of measuring the impact of individual data points in a cluster analysis is examined. The purpose is to identify those data points that have an influence on the resulting cluster partitions. Influence of a single data point is considered present when different cluster partitions result from the removal of the element from the data set. The Hubert and Arable corrected Rand index was used to provide numerical measures of influence of a data point. Simulated data sets consisting of a variety of cluster structures and error conditions were generated to validate the influence measures. The results showed that the measure of internal influence was 100% accurate in identifying those data elements exhibiting an influential effect. The nature of the influence, whether beneficial or detrimental to the clustering, can be evaluated with the use of the gamma and point-biserial statistics.

The Effect of Cluster Size, Dimensionality, and the Number of Clusters on Recovery of True Cluster Structure

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000

An evaluation of four clustering methods and four extemal criterion measures was conducted with r... more An evaluation of four clustering methods and four extemal criterion measures was conducted with respect to the effect of the number of clusters, dimensionality, and relative cluster sizes on the recovery of true cluster structure. The four methods were the single link, complete link, group average (UPGMA), and Ward's minimum variance algorithms. The results indicated that the four criterion measures were generally consistent with each other, of which two highly similar pairs were identified. The tirst pair consisted of the Rand and corrected Rand statistics, and the second pair was the Jaccard and the Fowlkes and Mallows indexes. With respect to the methods, recovery was found to improve as the number of clusters increased and as the number of dimensions increased. The relative cluster size factor produced differential perfonnance effects, with Ward's procedure providing the best recovery when the clusters were of equal size. The group average method gave equivalent or better recovery when the clusters were of unequal size.

Hierarchical Clustering Algorithms with Influence Detection

Educational and Psychological Measurement, 1995

Abstract The computer program described in this study is based on the methodology developed by th... more

K-Means Clustering Methods with Influence Detection

Educational and Psychological Measurement, 1996

Abstract A software program is described that performs K-means clustering with several enhancemen... more

A Two-Stage Clustering Algorithm with Robust Recovery Characteristics

Educational and Psychological Measurement, 1980

Two FORTRAN IV computer programs for a two-stage clustering algorithm with robust recovery charac... more Two FORTRAN IV computer programs for a two-stage clustering algorithm with robust recovery characteristics are described. In Stage 1, a group average hierarchical algorithm generates cluster centroids which are used as starting seed points for Stage 2, Jancey&amp;amp;amp;amp;#39;s k-means nonhierarchical algorithm. Also available with the hierarchical algorithm is a hypothesis test procedure which can be used to determine whether significant

An Algorithm for Calculating Coefficients Required for Trend Analysis

Educational and Psychological Measurement, 1980

A NOTE ON PROCEDURES FOR TESTING THE QUALITY OF A CLUSTERING OF A SET OF OBJECTS

Decision Sciences, 1980

Despite the increased application of cluster analysis in decision sciences, few attempts have bee... more Despite the increased application of cluster analysis in decision sciences, few attempts have been made to derive hypothesis-testing procedures for the evaluation of clustering solutions. In fact, the present paper shows that at least one such attempt failed to specify a meaningful sampling distribution for the test procedure. An alternative index based on the concept of point-biserial correlation is proposed as a possible recovery measure. The index is subsequently used to form the basis of a valid statistical test for the existence of cluster structure.

$Input Matrix of Direct Dissimilarities ——eo ws Oe eeeeeoerlcrOoOor rehome eee NOISES SEES “eae In order to test the null hypothesis of no cluster structure at each level in the hierarchy, 100 random-noise data sets were generated. A multivariate uniform distribution was used to generate the data. All variables were uncorrelated in the distribution. The random-noise sets contained the same number of elements as in the data set of interest. Clustering solutions for these data sets were obtained, and point-biserial correlations were computed for all cluster levels. Columns 4 and 5 in Table 3 give the mean and the standard deviation of the null distribution for each hierarchy level. Finally, the third column of Table 3 gives the level of signifi- cance. That is, the significance level indicates the fraction of the 100 random- noise data sets with values for the point-biserial correlation which exceeded that of the empirica] data set being analyzed.$

Methodology Review: Clustering Methods

Applied Psychological Measurement, 1987

A review of clustering methodology is presented, with emphasis on algorithm performance and the r... more A review of clustering methodology is presented, with emphasis on algorithm performance and the resulting implications for applied research. After an overview of the clustering literature, the clustering process is discussed within a seven-step framework. The four major types of clustering methods can be characterized as hierarchical, partitioning, overlapping, and ordination algorithms. The validation of such algorithms refers to the problem of determining the ability of the methods to recover cluster configurations which are known to exist in the data. Validation approaches include mathematical derivations, analyses of empirical datasets, and monte carlo simulation methods. Next, interpretation and inference procedures in cluster analysis are discussed. inference procedures involve testing for significant cluster structure and the problem of determining the number of clusters in the data. The paper concludes with two sets of recommendations.