Exploring Decomposition for Solving Pattern Mining Problems

Mining association rule is one of the key problems in data mining approach. Association rules discover the hidden relationships between various data items. In this paper, we propose a framework for the discovery of association rules using frequent pattern mining. We use preprocessing to transform the transaction dataset into a 2D matrix of 1's and 0's. Mining association rule must firstly discover frequent itemsets and then generate strong association rules from the frequent itemsets. The Apriori algorithm is the most well known association rule mining algorithm and is less efficient because they need to scan the database many times and store transaction ID in memory, so time and space overhead is very high. Especially they are less efficient when they process large scale database. Here we propose improved Apriori algorithm by including prune step and hash map data structure. The improved algorithm is more suitable for large scale database. Experimental results shows that computation times are reduced by using the prune step and hash map data structure.

International Journal of Applied Information Systems, 2015

Datasets grow in size as they are increasingly being gathered by cheap and numerous information-sensing mobile devices, aerial, software logs, microphones, wireless sensor networks and cameras. This paper presents a structure for simply, easily and competently parallelizing data mining algorithms for those huge datasets together with the incremental mining. MapReduce concept is use to execute the parallel FP-Growth algorithm by running the windows services parallel. The proposed algorithm eliminates duplicated work and spurious items. Also, it shortens the response time to a query for the set of frequent items. The proposed algorithm is implemented by parallel running of many windows services and experimental results shows tremendous advantages. The proposed algorithm runs 66% faster than the traditional algorithm of data mining. Also, memory utilization reduces by 37%.

2015

In this paper, we provide an overview of parallel incremental association rule mining, which is one of the imminent ideas in the new and rapidly emerging research area of data mining. A useful tool for discovering frequently co-occurrent items is frequent itemset mining (FIM). Since its commencement, a number of significant FIM algorithms have been build up to increase mining performance. But when thedataset size is huge, both the computational cost and memory use can be toocostly. In this paper,we put frontward parallelizing the FP-Growth algorithm.We use MapReduce to execute the parallelization of FP-Growth algorithm. Henceforth, it splits the mining task into number of sub-tasks, implements these sub-tasks in parallel on nodes and then combines the results back for the final result.Experiments show that the result increases the computational speed as compared to apriori and fp-growth. General Terms Data Mining, Association Rule Mining, Incremental Data Mining.

Data Mining and Knowledge Discovery, 2007

Frequent pattern mining has been a focused theme in data mining research for over a decade. Abundant literature has been dedicated to this research and tremendous progress has been made, ranging from efficient and scalable algorithms for frequent itemset mining in transaction databases to numerous research frontiers, such as sequential pattern mining, structured pattern mining, correlation mining, associative classification, and frequent pattern-based clustering, as well as their broad applications. In this article, we provide a brief overview of the current status of frequent pattern mining and discuss a few promising research directions. We believe that frequent pattern mining research has substantially broadened the scope of data analysis and will have deep impact on data mining methodologies and applications in the long run. However, there are still some challenging research issues that need to be solved before frequent pattern mining can claim a cornerstone approach in data mining applications.

International Journal of Information Technology and Computer Science, 2014

The process of data mining produces various patterns from a given data source. The most recognized data mining tasks are the process of discovering frequent itemsets, frequent sequential patterns, frequent sequential rules and frequent association rules. Numerous efficient algorithms have been proposed to do the above processes. Frequent pattern mining has been a focused topic in data mining research with a good number of references in literature and for that reason an important progress has been made, varying from performant algorithms for frequent itemset mining in transaction databases to complex algorithms, such as sequential pattern mining, structured pattern mining, correlation mining. Association Rule mining (ARM) is one of the utmost current data mining techniques designed to group objects together from large databases aiming to extract the interesting correlation and relation among huge amount of data. In this article, we provide a brief review and analysis of the current status of frequent pattern mining and discuss some promising research directions. Additionally, this paper includes a comparative study between the performance of the described approaches.

Australasian Conference on Knowledge Discovery and Data Mining, 2008

Rare association rule mining has received a great deal of attention in the recent past. In this research, we use transaction clustering as a pre-processing mech- anism to generate rare association rules. The basic concept underlying transaction clustering stems from the concept of large items as defined by traditional association rule mining algorithms. We make use of an approach proposed

2005 IEEE International Conference on Granular Computing, 2005

Mining frequent patterns is one of the fundamental and essential operations in many data mining applications, such as discovering association rules. In this paper, we propose an innovative approach to generating compact transaction databases for efficient frequent pattern mining. It uses a compact tree structure, called CT-tree, to compress the original transactional data. This allows the CT-Apriori algorithm, which is revised from the classical Apriori algorithm, to generate frequent patterns quickly by skipping the initial database scan and reducing a great amount of I/O time per database scan. Empirical evaluations show that our approach is effective, efficient and promising, while the storage space requirement as well as the mining time can be decreased dramatically on both synthetic and real-world databases.

Al-Azhar University Engineering Journal, JAUES, 11th International Conference, 2010

Clustering is an important data mining technique that groups similar data records, recently categorical transaction clustering is received more attention. In this research we study the problem of categorical data clustering for transactional data characterized with high dimensionality and large volume. We propose a novel algorithm for clustering transactional data called F-Tree, which is based on the idea of the frequent pattern algorithm FP-tree; the fastest approaches to frequent item set mining. And the simple idea behind the F-Tree is to generate small high pure clusters, and then merge them. That makes it fast, and dynamic in clustering large transactional datasets with high dimensions. We also present a new solution to solve the overlapping problem between clusters, by defining a new criterion function, which is based on the probability of overlapping between weighted items. Our experimental evaluation on real datasets shows that: Firstly, F-Tree is effective in finding interesting clusters. Secondly, the usage of the tree structure reduces the clustering process time of the large data set with high attributes. Thirdly, the proposed evaluation metric used efficiently to solve the overlapping of transaction items generates a high quality clustering results. Finally, we have concluded that the process of merging pure and small clusters increases the purity of resulted clusters as well as it reduces time of clustering better than the process of generating clusters directly from dataset then refine clusters.

International Journal of Computer Applications, 2014

Frequent pattern mining is a researched area which is used for extracting interesting associations and correlations among item sets in transactional and relational database. Many algorithms of frequent pattern mining is been devised ranging from efficient and scalable algorithms in transactional database to numerous research frontiers and their wide applications. Many researches been done into FPM [1], but there are still several optimizations are required, so that FPM can be used more efficiently in data mining applications. For optimization purpose in many mining techniques data pre-processing plays an important role in reducing data size and also in lessening the time taken in database scans. This paper is a detailed study of problems and solutions of FPM techniques incorporated with pre-processing techniques. The intent of this paper is to summarize all major problems of FPM and their solutions. From this survey, it concludes that if FPM methods are merged with pre-processing techniques will produce results with better performance.

The Journal of Supercomputing, 2013

In this paper we describe a new parallel Frequent Itemset Mining algorithm called "Frontier Expansion." This implementation is optimized to achieve high performance on a heterogeneous platform consisting of a shared memory multiprocessor and multiple Graphics Processing Unit (GPU) coprocessors. Frontier Expansion is an improved data-parallel algorithm derived from the Equivalent Class Clustering (Eclat) method, in which a partial breadth-first search is utilized to exploit maximum parallelism while being constrained by the available memory capacity. In our approach, the vertical transaction lists are represented using a "bitset" representation and operated using wide bitwise operations across multiple threads on a GPU. We evaluate our approach using four NVIDIA Tesla GPUs and observed a 6-30× speedup relative to state-of-the-art sequential Eclat and FPGrowth implementations executed on a multicore CPU.