International Journal of Pattern Recognition and Artificial Intelligence, Nov 1, 2007
In a classification problem, quite often the dimension of the measurement vector is large. Some o... more In a classification problem, quite often the dimension of the measurement vector is large. Some of these measurements may not be important for separating the classes. Removal of these measurement variables not only reduces the computational cost but also leads to better understanding of class separability. There are some methods in the existing literature for reducing the dimensionality of a classification problem without losing much of the separability information. However, these dimension reduction procedures usually work well for linear classifiers. In the case where competing classes are not linearly separable, one has to look for ideal "features" which could be some transformations of one or more measurements. In this paper, we make an attempt to tackle both, the problems of dimension reduction and feature extraction, by considering a projection pursuit regression model. The single hidden layer perceptron model and some other popular models can be viewed as special cases of this model. An iterative algorithm based on backfitting is proposed to select the features dynamically, and cross-validation method is used to select the ideal number of features. We carry out an extensive simulation study to show the effectiveness of this fully automatic method.
A very well-known traditional approach in discriminant analysis is to use some linear (or nonline... more A very well-known traditional approach in discriminant analysis is to use some linear (or nonlinear) combination of measurement variables which can enhance class separability. For instance, a linear (or a quadratic) classifier finds the linear projection (or the quadratic function) of the measurement variables that will maximize the separation between the classes. These techniques are very useful in obtaining good lower dimensional view of class separability. Fisher's discriminant analysis, which is primarily motivated by the multivariate normal distribution, uses the first-and second-order moments of the training sample to build such classifiers. These estimates, however, are highly sensitive to outliers, and they are not reliable for heavy-tailed distributions. This paper investigates two distribution-free methods for linear classification, which are based on the notions of statistical depth functions. One of these classifiers is closely related to Tukey's half-space depth, while the other is based on the concept of regression depth. Both these methods can be generalized for constructing nonlinear surfaces to discriminate among competing classes. These depth-based methods assume some finite-dimensional parametric form of the discriminating surface and use the distributional geometry of the data cloud to build the classifier. We use a few simulated and real data sets to examine the performance of these discriminant analysis tools and study their asymptotic properties under appropriate regularity conditions.
In this article, we propose some two-sample tests based on ball divergence and investigate their ... more In this article, we propose some two-sample tests based on ball divergence and investigate their high dimensional behavior. First, we study their behavior for High Dimension, Low Sample Size (HDLSS) data, and under appropriate regularity conditions, we establish their consistency in the HDLSS regime, where the dimension of the data grows to infinity while the sample sizes from the two distributions remain fixed. Further, we show that these conditions can be relaxed when the sample sizes also increase with the dimension, and in such cases, consistency can be proved even for shrinking alternatives. We use a simple example involving two normal distributions to prove that even when there are no consistent tests in the HDLSS regime, the powers of the proposed tests can converge to unity if the sample sizes increase with the dimension at an appropriate rate. This rate is obtained by establishing the minimax rate optimality of our tests over a certain class of alternatives. Several simulated and benchmark data sets are analyzed to compare the performance of these proposed tests with the state-of-the-art methods that can be used for testing the equality of two high-dimensional probability distributions.
Several parametric and nonparametric tests of independence between two random vectors are availab... more Several parametric and nonparametric tests of independence between two random vectors are available in the literature. But, many of them perform poorly for high dimensional data and are not applicable when the dimension exceeds the sample size. In this article, we propose some tests based on ranks of nearest neighbors, which can be conveniently used in high dimension, low sample size situations. Several simulated and real data sets are analyzed to show the utility of the proposed tests. Codes for implementation of the proposed tests are available as supplementary materials.
Journal of Statistical Planning and Inference, 2016
Abstract Several nonparametric methods are available in the literature to test the independence b... more Abstract Several nonparametric methods are available in the literature to test the independence between two random vectors. But, many of them perform poorly for high dimensional data and are not applicable when the dimension of one of these vectors exceeds the sample size. Moreover, most of these tests are not distribution-free in the general multivariate set up. Recently, Heller et al. (2012) proposed a test of independence, which is distribution-free and can be conveniently used even when the dimensions are larger than the sample size. In this article, we point out some limitations of this test and propose some modifications to overcome them retaining its distribution-free property. Some simulated and real data sets are analyzed to demonstrate the utility of our proposed modifications.
Testing for equality of two high-dimensional distributions is a challenging problem, and this bec... more Testing for equality of two high-dimensional distributions is a challenging problem, and this becomes even more challenging when the sample size is small. Over the last few decades, several graph-based two-sample tests have been proposed in the literature, which can be used for data of arbitrary dimensions. Most of these test statistics are computed using pairwise Euclidean distances among the observations. But, due to concentration of pairwise Euclidean distances, these tests have poor performance in many high-dimensional problems. Some of them can have powers even below the nominal level when the scale-difference between two distributions dominates the location-difference. To overcome these limitations, we introduce some new dissimilarity indices and use them to modify some popular graph-based tests. These modified tests use the distance concentration phenomenon to their advantage, and as a result, they outperform the corresponding tests based on the Euclidean distance in a wide variety of examples. We establish the high-dimensional consistency of these modified tests under fairly general conditions. Analyzing several simulated as well as real data sets, we demonstrate their usefulness in high dimension, low sample size situations.
There is a substantial literature on the estimation of error rate, or risk, for nonparametric cla... more There is a substantial literature on the estimation of error rate, or risk, for nonparametric classifiers. Error-rate estimation has at least two purposes: accurately describing the error rate, and estimating the tuning parameters that permit the error rate to be mininised. In the light of work on related problems in nonparametric statistics, it is attractive to argue that both problems admit the same solution. Indeed, methods for optimising the point-estimation performance of nonparametric curve estimators often start from an accurate estimator of error. However, we argue in this paper that accurate estimators of error rate in classification tend to give poor results when used to choose tuning parameters; and vice versa. Concise theory is used to illustrate this point in the case of cross-validation (which gives very accurate estimators of error rate, but poor estimators of tuning parameters) and the smoothed bootstrap (where error-rate estimation is poor but tuning-parameter approximations are particularly good). The theory is readily extended to other methods, for example to the 0.632+ bootstrap approach, which gives good estimators of error rate but poor estimators of tuning parameters. Reasons for the apparent contradiction are given, and numerical results are used to point to the practical implications of the theory.
In this article, we develop and investigate a new classifier based on features extracted using sp... more In this article, we develop and investigate a new classifier based on features extracted using spatial depth. Our construction is based on fitting a generalized additive model to the posterior probabilities of the different competing classes. To cope with possible multi-modal as well as non-elliptic population distributions, we develop a localized version of spatial depth and use that with varying degrees of localization to build the classifier. Final classification is done by aggregating several posterior probability estimates each of which is obtained using localized spatial depth with a fixed scale of localization. The proposed classifier can be conveniently used even when the dimension is larger than the sample size, and its good discriminatory power for such data has been established using theoretical as well as numerical results.
We propose some two-sample tests based on ball divergence and investigate their high dimensional ... more We propose some two-sample tests based on ball divergence and investigate their high dimensional behaviour. First, we consider the High Dimension, Low Sample Size (HDLSS) setup. Under appropriate regularity conditions, we establish the consistency of these tests in the HDLSS regime, where the dimension grows to infinity while the sample sizes from the two distributions remain fixed. Next, we show that these conditions can be relaxed when the sample sizes also increase with the dimension, and in such cases, consistency can be proved even for shrinking alternatives. We use a simple example to show that even when there are no consistent tests in the HDLSS regime, the proposed tests can be consistent if the sample sizes increase with the dimension at an appropriate rate. This rate is obtained by establishing the minimax rate optimality of these tests over a certain class of alternatives. Several simulated and benchmark data sets are analyzed to compare the empirical performance of these tests with some state-of-the-art methods available for testing the equality of two high-dimensional distributions.
In this article, we use L p depth for classification of multivariate data, where the value of p i... more In this article, we use L p depth for classification of multivariate data, where the value of p is chosen adaptively using observations from the training sample. While many depth based classifiers are constructed assuming elliptic symmetry of the underlying distributions, our proposed L p depth classifiers cater to a larger class of distributions. We establish Bayes risk consistency of these proposed classifiers under appropriate regularity conditions. Several simulated and benchmark data sets are analyzed to compare their finite sample performance with some existing parametric and nonparametric classifiers including those based on other notions of data depth.
Multivariate two-sample testing problem has been well investigated in the literature, and several... more Multivariate two-sample testing problem has been well investigated in the literature, and several parametric and nonparametric methods are available for it. However, most of these two-sample tests perform poorly for high dimensional data, and many of them are not applicable when the dimension of the data exceeds the sample size. In this article, we propose a nonparametric twosample test that can be conveniently used in the high dimension low sample size set up. Asymptotic results on the power properties of our proposed test are derived when the sample size remains fixed, and the dimension of the data grows to infinity. We investigate the performance of this test on several high dimensional simulated and real data sets and demonstrate its superiority over several other existing two-sample tests. We also study the theoretical properties of the proposed test for situations when the dimension of the data remains fixed and the sample size tends to infinity. In such cases, it turns out to be asymptotically distribution-free and consistent under general alternatives.
Over the last couple of decades, several copula based methods have been proposed in the literatur... more Over the last couple of decades, several copula based methods have been proposed in the literature to test for independence among several random variables. But these existing tests are not invariant under monotone transformations of the variables, and they often perform poorly if the dependence among the variables is highly non-monotone in nature. In this article, we propose a copula based measure of dependency and use it to construct some distribution-free tests of independence. The proposed measure and the resulting tests, all are invariant under permutations and strictly monotone transformations of the variables. Our dependency measure involves a kernel function with an associated bandwidth parameter. We adopt a multi-scale approach, where we look at the results obtained for several choices of the bandwidth and aggregate them judiciously. Large sample properties of the dependency measure and the resulting tests are derived under appropriate regularity conditions. Several simulated and real data sets are analyzed to compare the performance of the proposed tests with some popular tests available in the literature.
Testing for mutual independence among several random vectors is a challenging problem, and in rec... more Testing for mutual independence among several random vectors is a challenging problem, and in recent years, it has gained significant attention in statistics and machine learning literature. Most of the existing tests of independence deal with only two random vectors, and they do not have straightforward generalizations for testing mutual independence among more than two random vectors of arbitrary dimensions. On the other hand, there are various tests for mutual independence among several random variables, but these univariate tests do not have natural multivariate extensions. In this article, we propose two general recipes, one based on inter-point distances and the other based on linear projections, for multivariate extensions of these univariate tests. Under appropriate regularity conditions, these resulting tests turn out to be consistent whenever we have consistency for the corresponding univariate tests. We carry out extensive numerical studies to compare the empirical performance of these proposed methods with the state-of-the-art methods.
We use the ideas of maximum mean discrepancy and ranks of nearest neighbors to propose some tests... more We use the ideas of maximum mean discrepancy and ranks of nearest neighbors to propose some tests of independence among multiple random vectors of arbitrary dimensions. Numerical studies demonstrate that proposed tests can outperform the existing tests in various examples.
Journal of Statistical Computation and Simulation, Feb 4, 2021
Testing mutual independence among several random vectors of arbitrary dimensions is a challenging... more Testing mutual independence among several random vectors of arbitrary dimensions is a challenging problem in Statistics, and it has gained considerable interest in recent years. In this article, we propose some nonparametric tests based on different notions of ranks of nearest neighbour. These proposed tests can be conveniently used for high dimensional data, even when the dimensions of the random vectors are larger than the sample size. We investigate the performance of these tests on several simulated and real data sets and also use them in identifying causal relationships among the random vectors. Our numerical results show that they can outperform state-of-the-art tests in a wide variety of examples.
When the competing classes in a classification problem are not of comparable size, many popular c... more When the competing classes in a classification problem are not of comparable size, many popular classifiers exhibit a bias towards larger classes, and the nearest neighbor classifier is no exception. To take care of this problem, we develop a statistical method for nearest neighbor classification based on such imbalanced data sets. First, we construct a classifier for the binary classification problem and then extend it for classification problems involving more than two classes. Unlike the existing oversampling or undersampling methods, our proposed classifiers do not need to generate any pseudo observations or remove any existing observations, hence the results are exactly reproducible. We establish the Bayes risk consistency of these classifiers under appropriate regularity conditions. Their superior performance over the existing methods is amply demonstrated by analyzing several benchmark data sets.
We propose a new model-free feature screening method based on energy distances for ultrahigh-dime... more We propose a new model-free feature screening method based on energy distances for ultrahigh-dimensional binary classification problems. Unlike existing methods, the cutoff involved in our procedure is data adaptive. With a high probability, the proposed method retains only relevant features after discarding all the noise variables. The proposed screening method is also extended to identify pairs of variables that are marginally undetectable, but have differences in their joint distributions. Finally, we build a classifier which maintains coherence between the proposed feature selection criteria and discrimination method, and also establish its risk consistency. An extensive numerical study with simulated data sets and real benchmark data sets show clear and convincing advantages of our classifier over the state-of-the-art methods.
Over the last couple of decades, several copula based methods have been proposed in the literatur... more Over the last couple of decades, several copula based methods have been proposed in the literature to test for independence among several random variables. But these existing tests are not invariant under monotone transformations of the variables, and they often perform poorly if the dependence among the variables is highly non-monotone in nature. In this article, we propose a copula based measure of dependency and use it to construct some distribution-free tests of independence. The proposed measure and the resulting tests, all are invariant under permutations and strictly monotone transformations of the variables. Our dependency measure involves a kernel function with an associated bandwidth parameter. We adopt a multi-scale approach, where we look at the results obtained for several choices of the bandwidth and aggregate them judiciously. Large sample properties of the dependency measure and the resulting tests are derived under appropriate regularity conditions. Several simulated and real data sets are analyzed to compare the performance of the proposed tests with some popular tests available in the literature.
In high dimension, low sample size (HDLSS) settings, Euclidean distance based classifiers suffer ... more In high dimension, low sample size (HDLSS) settings, Euclidean distance based classifiers suffer from curse of dimensionality if the competing populations are similar in terms of their location and scale parameters. In this article, we propose a classifier which can discriminate between populations having different marginal distributions. A generalization of the proposed classifier has been developed to differentiate between populations having different joint distributions for the component variables. Performance of the proposed classifiers are demonstrated using a variety of simulated data sets.
International Journal of Pattern Recognition and Artificial Intelligence, Nov 1, 2007
In a classification problem, quite often the dimension of the measurement vector is large. Some o... more In a classification problem, quite often the dimension of the measurement vector is large. Some of these measurements may not be important for separating the classes. Removal of these measurement variables not only reduces the computational cost but also leads to better understanding of class separability. There are some methods in the existing literature for reducing the dimensionality of a classification problem without losing much of the separability information. However, these dimension reduction procedures usually work well for linear classifiers. In the case where competing classes are not linearly separable, one has to look for ideal "features" which could be some transformations of one or more measurements. In this paper, we make an attempt to tackle both, the problems of dimension reduction and feature extraction, by considering a projection pursuit regression model. The single hidden layer perceptron model and some other popular models can be viewed as special cases of this model. An iterative algorithm based on backfitting is proposed to select the features dynamically, and cross-validation method is used to select the ideal number of features. We carry out an extensive simulation study to show the effectiveness of this fully automatic method.
A very well-known traditional approach in discriminant analysis is to use some linear (or nonline... more A very well-known traditional approach in discriminant analysis is to use some linear (or nonlinear) combination of measurement variables which can enhance class separability. For instance, a linear (or a quadratic) classifier finds the linear projection (or the quadratic function) of the measurement variables that will maximize the separation between the classes. These techniques are very useful in obtaining good lower dimensional view of class separability. Fisher's discriminant analysis, which is primarily motivated by the multivariate normal distribution, uses the first-and second-order moments of the training sample to build such classifiers. These estimates, however, are highly sensitive to outliers, and they are not reliable for heavy-tailed distributions. This paper investigates two distribution-free methods for linear classification, which are based on the notions of statistical depth functions. One of these classifiers is closely related to Tukey's half-space depth, while the other is based on the concept of regression depth. Both these methods can be generalized for constructing nonlinear surfaces to discriminate among competing classes. These depth-based methods assume some finite-dimensional parametric form of the discriminating surface and use the distributional geometry of the data cloud to build the classifier. We use a few simulated and real data sets to examine the performance of these discriminant analysis tools and study their asymptotic properties under appropriate regularity conditions.
In this article, we propose some two-sample tests based on ball divergence and investigate their ... more In this article, we propose some two-sample tests based on ball divergence and investigate their high dimensional behavior. First, we study their behavior for High Dimension, Low Sample Size (HDLSS) data, and under appropriate regularity conditions, we establish their consistency in the HDLSS regime, where the dimension of the data grows to infinity while the sample sizes from the two distributions remain fixed. Further, we show that these conditions can be relaxed when the sample sizes also increase with the dimension, and in such cases, consistency can be proved even for shrinking alternatives. We use a simple example involving two normal distributions to prove that even when there are no consistent tests in the HDLSS regime, the powers of the proposed tests can converge to unity if the sample sizes increase with the dimension at an appropriate rate. This rate is obtained by establishing the minimax rate optimality of our tests over a certain class of alternatives. Several simulated and benchmark data sets are analyzed to compare the performance of these proposed tests with the state-of-the-art methods that can be used for testing the equality of two high-dimensional probability distributions.
Several parametric and nonparametric tests of independence between two random vectors are availab... more Several parametric and nonparametric tests of independence between two random vectors are available in the literature. But, many of them perform poorly for high dimensional data and are not applicable when the dimension exceeds the sample size. In this article, we propose some tests based on ranks of nearest neighbors, which can be conveniently used in high dimension, low sample size situations. Several simulated and real data sets are analyzed to show the utility of the proposed tests. Codes for implementation of the proposed tests are available as supplementary materials.
Journal of Statistical Planning and Inference, 2016
Abstract Several nonparametric methods are available in the literature to test the independence b... more Abstract Several nonparametric methods are available in the literature to test the independence between two random vectors. But, many of them perform poorly for high dimensional data and are not applicable when the dimension of one of these vectors exceeds the sample size. Moreover, most of these tests are not distribution-free in the general multivariate set up. Recently, Heller et al. (2012) proposed a test of independence, which is distribution-free and can be conveniently used even when the dimensions are larger than the sample size. In this article, we point out some limitations of this test and propose some modifications to overcome them retaining its distribution-free property. Some simulated and real data sets are analyzed to demonstrate the utility of our proposed modifications.
Testing for equality of two high-dimensional distributions is a challenging problem, and this bec... more Testing for equality of two high-dimensional distributions is a challenging problem, and this becomes even more challenging when the sample size is small. Over the last few decades, several graph-based two-sample tests have been proposed in the literature, which can be used for data of arbitrary dimensions. Most of these test statistics are computed using pairwise Euclidean distances among the observations. But, due to concentration of pairwise Euclidean distances, these tests have poor performance in many high-dimensional problems. Some of them can have powers even below the nominal level when the scale-difference between two distributions dominates the location-difference. To overcome these limitations, we introduce some new dissimilarity indices and use them to modify some popular graph-based tests. These modified tests use the distance concentration phenomenon to their advantage, and as a result, they outperform the corresponding tests based on the Euclidean distance in a wide variety of examples. We establish the high-dimensional consistency of these modified tests under fairly general conditions. Analyzing several simulated as well as real data sets, we demonstrate their usefulness in high dimension, low sample size situations.
There is a substantial literature on the estimation of error rate, or risk, for nonparametric cla... more There is a substantial literature on the estimation of error rate, or risk, for nonparametric classifiers. Error-rate estimation has at least two purposes: accurately describing the error rate, and estimating the tuning parameters that permit the error rate to be mininised. In the light of work on related problems in nonparametric statistics, it is attractive to argue that both problems admit the same solution. Indeed, methods for optimising the point-estimation performance of nonparametric curve estimators often start from an accurate estimator of error. However, we argue in this paper that accurate estimators of error rate in classification tend to give poor results when used to choose tuning parameters; and vice versa. Concise theory is used to illustrate this point in the case of cross-validation (which gives very accurate estimators of error rate, but poor estimators of tuning parameters) and the smoothed bootstrap (where error-rate estimation is poor but tuning-parameter approximations are particularly good). The theory is readily extended to other methods, for example to the 0.632+ bootstrap approach, which gives good estimators of error rate but poor estimators of tuning parameters. Reasons for the apparent contradiction are given, and numerical results are used to point to the practical implications of the theory.
In this article, we develop and investigate a new classifier based on features extracted using sp... more In this article, we develop and investigate a new classifier based on features extracted using spatial depth. Our construction is based on fitting a generalized additive model to the posterior probabilities of the different competing classes. To cope with possible multi-modal as well as non-elliptic population distributions, we develop a localized version of spatial depth and use that with varying degrees of localization to build the classifier. Final classification is done by aggregating several posterior probability estimates each of which is obtained using localized spatial depth with a fixed scale of localization. The proposed classifier can be conveniently used even when the dimension is larger than the sample size, and its good discriminatory power for such data has been established using theoretical as well as numerical results.
We propose some two-sample tests based on ball divergence and investigate their high dimensional ... more We propose some two-sample tests based on ball divergence and investigate their high dimensional behaviour. First, we consider the High Dimension, Low Sample Size (HDLSS) setup. Under appropriate regularity conditions, we establish the consistency of these tests in the HDLSS regime, where the dimension grows to infinity while the sample sizes from the two distributions remain fixed. Next, we show that these conditions can be relaxed when the sample sizes also increase with the dimension, and in such cases, consistency can be proved even for shrinking alternatives. We use a simple example to show that even when there are no consistent tests in the HDLSS regime, the proposed tests can be consistent if the sample sizes increase with the dimension at an appropriate rate. This rate is obtained by establishing the minimax rate optimality of these tests over a certain class of alternatives. Several simulated and benchmark data sets are analyzed to compare the empirical performance of these tests with some state-of-the-art methods available for testing the equality of two high-dimensional distributions.
In this article, we use L p depth for classification of multivariate data, where the value of p i... more In this article, we use L p depth for classification of multivariate data, where the value of p is chosen adaptively using observations from the training sample. While many depth based classifiers are constructed assuming elliptic symmetry of the underlying distributions, our proposed L p depth classifiers cater to a larger class of distributions. We establish Bayes risk consistency of these proposed classifiers under appropriate regularity conditions. Several simulated and benchmark data sets are analyzed to compare their finite sample performance with some existing parametric and nonparametric classifiers including those based on other notions of data depth.
Multivariate two-sample testing problem has been well investigated in the literature, and several... more Multivariate two-sample testing problem has been well investigated in the literature, and several parametric and nonparametric methods are available for it. However, most of these two-sample tests perform poorly for high dimensional data, and many of them are not applicable when the dimension of the data exceeds the sample size. In this article, we propose a nonparametric twosample test that can be conveniently used in the high dimension low sample size set up. Asymptotic results on the power properties of our proposed test are derived when the sample size remains fixed, and the dimension of the data grows to infinity. We investigate the performance of this test on several high dimensional simulated and real data sets and demonstrate its superiority over several other existing two-sample tests. We also study the theoretical properties of the proposed test for situations when the dimension of the data remains fixed and the sample size tends to infinity. In such cases, it turns out to be asymptotically distribution-free and consistent under general alternatives.
Over the last couple of decades, several copula based methods have been proposed in the literatur... more Over the last couple of decades, several copula based methods have been proposed in the literature to test for independence among several random variables. But these existing tests are not invariant under monotone transformations of the variables, and they often perform poorly if the dependence among the variables is highly non-monotone in nature. In this article, we propose a copula based measure of dependency and use it to construct some distribution-free tests of independence. The proposed measure and the resulting tests, all are invariant under permutations and strictly monotone transformations of the variables. Our dependency measure involves a kernel function with an associated bandwidth parameter. We adopt a multi-scale approach, where we look at the results obtained for several choices of the bandwidth and aggregate them judiciously. Large sample properties of the dependency measure and the resulting tests are derived under appropriate regularity conditions. Several simulated and real data sets are analyzed to compare the performance of the proposed tests with some popular tests available in the literature.
Testing for mutual independence among several random vectors is a challenging problem, and in rec... more Testing for mutual independence among several random vectors is a challenging problem, and in recent years, it has gained significant attention in statistics and machine learning literature. Most of the existing tests of independence deal with only two random vectors, and they do not have straightforward generalizations for testing mutual independence among more than two random vectors of arbitrary dimensions. On the other hand, there are various tests for mutual independence among several random variables, but these univariate tests do not have natural multivariate extensions. In this article, we propose two general recipes, one based on inter-point distances and the other based on linear projections, for multivariate extensions of these univariate tests. Under appropriate regularity conditions, these resulting tests turn out to be consistent whenever we have consistency for the corresponding univariate tests. We carry out extensive numerical studies to compare the empirical performance of these proposed methods with the state-of-the-art methods.
We use the ideas of maximum mean discrepancy and ranks of nearest neighbors to propose some tests... more We use the ideas of maximum mean discrepancy and ranks of nearest neighbors to propose some tests of independence among multiple random vectors of arbitrary dimensions. Numerical studies demonstrate that proposed tests can outperform the existing tests in various examples.
Journal of Statistical Computation and Simulation, Feb 4, 2021
Testing mutual independence among several random vectors of arbitrary dimensions is a challenging... more Testing mutual independence among several random vectors of arbitrary dimensions is a challenging problem in Statistics, and it has gained considerable interest in recent years. In this article, we propose some nonparametric tests based on different notions of ranks of nearest neighbour. These proposed tests can be conveniently used for high dimensional data, even when the dimensions of the random vectors are larger than the sample size. We investigate the performance of these tests on several simulated and real data sets and also use them in identifying causal relationships among the random vectors. Our numerical results show that they can outperform state-of-the-art tests in a wide variety of examples.
When the competing classes in a classification problem are not of comparable size, many popular c... more When the competing classes in a classification problem are not of comparable size, many popular classifiers exhibit a bias towards larger classes, and the nearest neighbor classifier is no exception. To take care of this problem, we develop a statistical method for nearest neighbor classification based on such imbalanced data sets. First, we construct a classifier for the binary classification problem and then extend it for classification problems involving more than two classes. Unlike the existing oversampling or undersampling methods, our proposed classifiers do not need to generate any pseudo observations or remove any existing observations, hence the results are exactly reproducible. We establish the Bayes risk consistency of these classifiers under appropriate regularity conditions. Their superior performance over the existing methods is amply demonstrated by analyzing several benchmark data sets.
We propose a new model-free feature screening method based on energy distances for ultrahigh-dime... more We propose a new model-free feature screening method based on energy distances for ultrahigh-dimensional binary classification problems. Unlike existing methods, the cutoff involved in our procedure is data adaptive. With a high probability, the proposed method retains only relevant features after discarding all the noise variables. The proposed screening method is also extended to identify pairs of variables that are marginally undetectable, but have differences in their joint distributions. Finally, we build a classifier which maintains coherence between the proposed feature selection criteria and discrimination method, and also establish its risk consistency. An extensive numerical study with simulated data sets and real benchmark data sets show clear and convincing advantages of our classifier over the state-of-the-art methods.
Over the last couple of decades, several copula based methods have been proposed in the literatur... more Over the last couple of decades, several copula based methods have been proposed in the literature to test for independence among several random variables. But these existing tests are not invariant under monotone transformations of the variables, and they often perform poorly if the dependence among the variables is highly non-monotone in nature. In this article, we propose a copula based measure of dependency and use it to construct some distribution-free tests of independence. The proposed measure and the resulting tests, all are invariant under permutations and strictly monotone transformations of the variables. Our dependency measure involves a kernel function with an associated bandwidth parameter. We adopt a multi-scale approach, where we look at the results obtained for several choices of the bandwidth and aggregate them judiciously. Large sample properties of the dependency measure and the resulting tests are derived under appropriate regularity conditions. Several simulated and real data sets are analyzed to compare the performance of the proposed tests with some popular tests available in the literature.
In high dimension, low sample size (HDLSS) settings, Euclidean distance based classifiers suffer ... more In high dimension, low sample size (HDLSS) settings, Euclidean distance based classifiers suffer from curse of dimensionality if the competing populations are similar in terms of their location and scale parameters. In this article, we propose a classifier which can discriminate between populations having different marginal distributions. A generalization of the proposed classifier has been developed to differentiate between populations having different joint distributions for the component variables. Performance of the proposed classifiers are demonstrated using a variety of simulated data sets.
Uploads
Papers by Anil Ghosh