CLUSTER ANALYSIS: A SURVEY

International Journal of Computer Applications, 2013

Data Mining is used to extract useful information from a collection of databases or data warehouses. In recent years, Data Mining has become an important field. This paper has surveyed upon data mining and its various techniques that are used to extract useful information such as clustering, and has also surveyed the techniques that are used to detect the outliers. This paper also presents various techniques used by different researchers to detect outliers and present the efficient result to the user.

International Journal of Advance Research and Innovative Ideas in Education, 2017

Data mining has the crucial task of Outlier detection which aims to detect an outlier from given data set. The data is said to be an outlier which appears to have inconsistent observation with the remaining data. Outliers are generated because of improper measurements, data entry errors or data arriving from various sources than remaining data. Outlier detection is the technique which discovers such type of data from the given data set. Several techniques of outlier detection have been introduced which requires input parameter from the user such as distance threshold, density threshold, etc. The goal of this proposed work is to partition the input data set into the number of clusters using Unsupervised Extreme Learning Machine algorithm. Then the clusters are given as an input to the outlier detection methods namely cluster based outlier detection algorithm and outlier detection algorithm. The methods detects an outlier from each cluster. This work aims at studying cluster based out...

Data mining, in general, deals with the discovery of non-trivial, hidden and interesting knowledge from different types of data. With the development of information technologies, the number of databases, as well as their dimension and complexity, grow rapidly. It is necessary what we need automated analysis of great amount of information. The analysis results are then used for making a decision by a human or program. One of the basic problems of data mining is the outlier detection. The outlier detection problem in some cases is similar to the classification problem. For example, the main concern of clustering-based outlier detection algorithms is to find clusters and outliers, which are often regarded as noise that should be removed in order to make more reliable clustering. In this thesis, the ability to detect outliers can be improved using a combined perspective from outlier detection and cluster identification. In proposed work comparison of four methods will be done like K-Mean, k-Mediods, Iterative k-Mean and density based method. Unlike the traditional clustering-based methods, the proposed algorithm provides much efficient outlier detection and data clustering capabilities in the presence of outliers, so comparison has been made. The purpose of our method is not only to produce data clustering but at the same time to find outliers from the resulting clusters. The goal is to model an unknown nonlinear function based on observed input-output pairs. The whole simulation of this proposed work has been taken in MATLAB environment.

Data mining is the extraction of hidden predictive information from large databases. This is a technology with potential to study and analyze useful information present in data. Data objects which do not usually fit into the general behavior of the data are termed as outliers. Outlier Detection in databases has numerous applications such as fraud detection, customized marketing, and the search for terrorism. By definition, outliers are rare occurrences and hence represent a small portion of the data. However, the use of Outlier Detection for various purposes is not an easy task. This research proposes a modified PAM for detecting outliers. The proposed technique has been implemented in JAVA. The results produced by the proposed technique are found better than existing technique in terms of outliers detected and time complexity.

2011 Second International Conference on Emerging Applications of Information Technology, 2011

In this paper we propose a clustering based method to capture outliers. We apply K-means clustering algorithm to divide the data set into clusters. The points which are lying near the centroid of the cluster are not probable candidate for outlier and we can prune out such points from each cluster. Next we calculate a distance based outlier score for remaining points. The computations needed to calculate the outlier score reduces considerably due to the pruning of some points. Based on the outlier score we declare the top n points with the highest score as outliers. The experimental results using real data set demonstrate that even though the number of computations is less, the proposed method performs better than the existing method.

In modern era there are lots of data mining algorithms which focus on clustering methods. There are also several types of approaches designed for outlier detection. Outliers are those data objects that do not fulfill with the common behavior or model of the data. Many data mining algorithms try to reduce the effects of outliers or remove them all together. We investigated that in many different conditions clusters and outliers whose meanings are connected to each other, especially for those data sets which contains some noise. So it is important to deal clusters and outliers as concepts of the same significance in data analysis. So in this paper we introduce an algorithm which is based on k means [1] for the detection of clusters and outliers that aim to detect the clusters and the outliers in a different view for those data sets which contains some noise. In this algorithm clusters are detected and managed according to the intra-relationship within the clusters and interrelationship between the clusters and the outliers. The whole management and modification of the clusters and outliers are done repeatedly just before a certain termination is reached.

— Outlier is defined as an event that deviates too much from other events. The identification of outlier can lead to the discovery of useful and meaningful knowledge. Outlier means it's happen at some time it's not regular activity. Research about Detection of Outlier has been extensively studies in the past decade. However, most existing research focused on the algorithm based on specific knowledge, compared with outlier detection approach is still rare. In this paper mainly focused on different kind of outlier detection approaches and compares it's prone and cones. In this paper we mainly distribute of outlier detection approach in two parts classic outlier approach and spatial outlier approach. The classical outlier approach identifies outlier in real transaction dataset, which can be grouped into statistical approach, distance approach, deviation approach, and density approach. The spatial outlier approach detect outlier based on spatial dataset are different from transaction data, which can be categorized into spaced approach and graph approach. Finally, the comparison of outlier detection approaches.

International Journal of Database Theory and Application, 2016

In this paper, the impact of-means and local outliner factor on data set is studied. Outlier is the observation which is different from or inconsistent with the rest of the data. However, the main challenges of outlier detection are increasing complexity due to variety of datasets and size of dataset. To evaluate the outlierness and catch similar outliers as a group are also issues of this technique. The concept of (Local Outlier Factor) is presented in this work. The paper describes comparative study of five different methodologies using-means as the base algorithm along with the various distances method used in finding the dissimilarities between the objects hence to analyze the effects of the outliers on the cluster analysis of dataset in data mining.

2015

In data mining outlier detection refers to the recognition of data point which does not follow the expected pattern or behavior in a particular dataset or is significantly different from other points in a data. In this paper we will review some of the outlier detection techniques and discuss their advantages and disadvantages with respect to various aspects. Outlier detection techniques can be classified into three modes namely unsupervised, semi-supervised and supervised. But, unsupervised outlier detection methods can be further classified as distance based or density based. Many outlier detection techniques are proposed till date. These proposed techniques can be broadly categorized as distribution based (statistical), clustering-based, density-based and model-based

2010

Outlier detection is an important task in a wide variety of application areas. In this paper, a proposed method based on fuzzy clustering approaches for outlier detection is presented. We first perform the c-means fuzzy clustering algorithm. Small clusters are then determined and considered as outlier clusters. The rest of outliers (if any) are then detected in the remaining clusters based on temporary removing a point from the data set and recalculating the objective function. If a noticeable change occurred in the Objective Function (OF), the point is considered an outlier. Experimental results show that our method works well. The test results show that the proposed approach gave good results when applied to different data sets.