PageRank is a basic metric used to measure the importance of web pages. It is an iterative method... more PageRank is a basic metric used to measure the importance of web pages. It is an iterative method, and computed by Power method. Power method converges too slowly for the true structure of web. This issues encourages to find out some other iterative methods to accelerate PageRank computation. In this paper, we present a novel method that is combination of Quadratic Extrapolation and Gauss-Seidel called QE-GSeidel method which accelerate the convergence of PageRank method. From numerical experiment we show that QE-GSeidel method takes almost 60-65% less number of iteration and 25-30% less CPU time than Power, and takes 50-55% less number of iteration and 15-20% less CPU time than Gauss-Seidel in PageRank computation.
International Journal of Computer Applications, 2010
Association Rule Mining from transaction-oriented databases is one of the important process that ... more Association Rule Mining from transaction-oriented databases is one of the important process that finds relation between items and plays important role in decision making. Parallel algorithms are required because of large size of the database to be mined. Most of the algorithms designed were for homogeneous system uses static load balancing technique which is far from reality. A parallel algorithm for heterogeneous system is regarded as one of the most promising platforms for association rule mining. In this paper we propose a simple parallel algorithm for association rule mining on heterogeneous system with dynamic load balancing based on Par-Maxclique algorithm. We maintain one linked list at the scheduler end that keep track of load value of every processor and each processor is having a job queue associated with it which is served in First come first basis. On the basis of load value scheduler directs the migration of task from heavy loaded to least loaded processor in the cluster during the execution and thus balances load dynamically in a cluster.
2009 International Conference on Computer and Automation Engineering, 2009
... Algorithms Rakhi Garg, PK Mishra Department Of Computer Science, Banaras Hindu University, Va... more ... Algorithms Rakhi Garg, PK Mishra Department Of Computer Science, Banaras Hindu University, Varanasi, India e-mail: [email protected] ,[email protected] ... A. APRIORI, Sequential ARM algorithm -Proposed by Rakesh Agrawal and J. Shafer [14]. ...
Association Rule Mining (ARM) is one of the well know and most researched technique of data minin... more Association Rule Mining (ARM) is one of the well know and most researched technique of data mining. There are so many ARM algorithms have been designed that their counting is a large number. In this paper we have surveyed the various ARM algorithms in four computing environments. The considered computing environments are sequential computing, parallel and distributed computing, grid computing and cloud computing. With the emergence of new computing paradigm, ARM algorithms have been designed by many researchers to improve the efficiency by utilizing the new paradigm. This paper represents the journey of ARM algorithms started from sequential algorithms, and through parallel and distributed, and grid based algorithms to the current state-of-the-art, along with the motives for adopting new machinery.
Mining frequent itemsets from massive datasets is always being a most important problem of data m... more Mining frequent itemsets from massive datasets is always being a most important problem of data mining. Apriori is the most popular and simplest algorithm for frequent itemset mining. To enhance the efficiency and scalability of Apriori, a number of algorithms have been proposed addressing the design of efficient data structures, minimizing database scan and parallel and distributed processing. MapReduce is the emerging parallel and distributed technology to process big datasets on Hadoop Cluster. To mine big datasets it is essential to re-design the data mining algorithm on this new paradigm. In this paper, we implement three variations of Apriori algorithm using data structures hash tree, trie and hash table trie i.e. trie with hash technique on MapReduce paradigm. We emphasize and investigate the significance of these three data structures for Apriori algorithm on Hadoop cluster, which has not been given attention yet. Experiments are carried out on both real life and synthetic d...
Due to huge, unstructured and scattered amount of data available on web, it is very tough for use... more Due to huge, unstructured and scattered amount of data available on web, it is very tough for users to get relevant information in less time. To achieve this, improvement in design of web site, personalization of contents, prefetching and caching activities are done according to user’s behavior analysis. User’s activities can be captured into a special file called log file. There are various types of log: Server log, Proxy server log, Client/Browser log. These log files are used by web usage mining to analyze and discover useful patterns. The process of web usage mining involves three interdependent steps: Data preprocessing, Pattern discovery and Pattern analysis. Among these steps, Data preprocessing plays a vital role because of unstructured, redundant and noisy nature of log data. To improve later phases of web usage mining like Pattern discovery and Pattern analysis several data preprocessing techniques such as Data Cleaning, User Identification, Session Identification, Path Co...
Data cleaning plays an important role in Web usage mining process as it reduces the complexity of... more Data cleaning plays an important role in Web usage mining process as it reduces the complexity of later phases of KDD (Knowledge Discovery in Databases) process. It incorporates the removal of those log entries and fields that are not relevant for the purpose of Web usage mining. Generally, Data cleaning includes the removal of unsuccessful requests, irrelevant files’ requests, inappropriate access methods’ requests, and robots’ requests. With the enormous increase of log data, traditional Data cleaning algorithm has become highly data intensive task. In the last few years, MapReduce has emerged as most widely used parallel programming framework to compute data intensive application on a cluster of nodes. In this article, we have proposed a MapReduce-based Parallel Data cleaning (PDC) algorithm that considers all aspect of the traditional Data cleaning algorithm. There are two modifications to exiting Data cleaning algorithm i.e. first is an efficient robot detection approach, and o...
Association Rule Mining (ARM) is one of the well know and most researched technique of data minin... more Association Rule Mining (ARM) is one of the well know and most researched technique of data mining. There are so many ARM algorithms have been designed that their counting is a large number. In this paper we have surveyed the various ARM algorithms in four computing environments. The considered computing environments are sequential computing, parallel and distributed computing, grid computing and cloud computing. With the emergence of new computing paradigm, ARM algorithms have been designed by many researchers to improve the efficiency by utilizing the new paradigm. This paper represents the journey of ARM algorithms started from sequential algorithms, and through parallel and distributed, and grid based algorithms to the current state-of-the-art, along with the motives for adopting new machinery.
International Journal of Advanced Research in Computer Science, 2017
Due to proliferation of Web robots, it is becoming important to detect robots on commercial and e... more Due to proliferation of Web robots, it is becoming important to detect robots on commercial and educational websites. Web robots not only consumes a significant proportion of network bandwidth but also a threat for privacy and security. There are various techniques to detect robot from server log i.e. checking robots.txt access, matching keywords in User agent field, calculating browsing speed etc. In this paper, we have implemented and compared theabove three robot detection approaches on real Web server log. Moreover, we have also applied combined approach to detect robots and found that combined approach works better than other three approaches.
Association rule mining is a time consuming process due to involving both data intensive and comp... more Association rule mining is a time consuming process due to involving both data intensive and computation intensive nature. In order to mine large volume of data and to enhance the scalability and performance of existing sequential association rule mining algorithms, parallel and distributed algorithms are developed. These traditional parallel and distributed algorithms are based on homogeneous platform and are not lucrative for heterogeneous platform such as grid and cloud. This requires design of new algorithms which address the issues of good data set partition and distribution, load balancing strategy, optimization of communication and synchronization technique among processors in such heterogeneous system. Grid and cloud are the emerging platform for distributed data processing and various association rule mining algorithms have been proposed on such platforms. This survey article integrates the brief architectural aspect of distributed system, various recent approaches of grid ...
72 Abstract— Among various data mining task, association rule mining (ARM) is the major technique... more 72 Abstract— Among various data mining task, association rule mining (ARM) is the major technique which is widely used in retail marketing, bioinformatics, website navigation analysis etc. It finds correlations among items in a given data sets and establishes an association between two non overlapping sets of frequently occurring values in a database. Various sequential and parallel ARM algorithms have been developed that differs in data layout, search technique, data structure, the number of database scans used and the system on which it is developed i.e. homogeneous or heterogeneous systems. This paper mainly put emphasis in the need of a candidate based ARM algorithm for heterogeneous PC cluster that reduces the database scans and time complexity. It also describe the design and functioning of the heterogeneous PC cluster i.e. MPICH2 and the data decomposition technique applied for candidate itemsets generation that plays important role in balancing workload as well as enhancing ...
Data preprocessing is an essential task to prepare suitable target datasets to apply statistical ... more Data preprocessing is an essential task to prepare suitable target datasets to apply statistical and data mining algorithms. It has become one of the complex segment in Web usage mining due to the ...
Association rule mining is one of the major technique of data mining, involves finding of frequen... more Association rule mining is one of the major technique of data mining, involves finding of frequent itemsets with minimum support and generating association rule among them with minimum confidence. The task of finding all frequent itemsets for a large datasets requires a lot of computation which can be minimized by exploiting parallelism to the sequential algorithms. In this paper, we provide the preliminaries of basic concepts about association rule mining, different sequential association rule mining algorithms on different hardware platforms and also focus on the challenges in exploiting parallelism to these algorithms. We also discusses up to what extent these challenges e.g. load balancing, efficient memory usage, minimization of communication cost among processors, efficient data and task decomposition etc. are congregate by a given parallel association rule mining algorithm and classifies them accordingly. Although, this survey cannot be complete review of all algorithms, but ...
PageRank is one of the basic metric used in web search technology to rank the web pages. It uses ... more PageRank is one of the basic metric used in web search technology to rank the web pages. It uses power method to compute principal eigenvector of the web matrix of several billion nodes. PageRank method incorporates a parameter α called damping factor that plays a major role in PageRank computation. In this study, we have observed experimentally the efficiency of various iterative methods on hyperlink graph for different value of α. We conclude from experiment that Power method is effective and more competitive for the well condition problem i.e. small value of α. However, for α → 1 Power method becomes more complex, and other methods such as Aitken-Power, SOR, and Gauss-Seidel are more efficient than it in respect of CPU time as well as the number of iteration needed for convergence.
Big Data is an excessive amount of imprecise data in variety of formats generated from variety of... more Big Data is an excessive amount of imprecise data in variety of formats generated from variety of sources with rapid speed. It is most buzzed terms among researcher, industry and academia. Big Data is not only limited to data perspective but it has been emerged as a stream that includes associated technologies, tools and real word applications. The objective of this paper is to provide a simple, comprehensive and brief introduction of Big Data to the beginners in subject. In this paper, we provide an overview of Hadoop and its subprojects and a brief review of various developed technologies for Big Data. We also discuss some recent trends and eminent applications in Big Data. Although this paper does not touch each and every dimension of Big Data as it is not possible to make it in a single paper but essential aspects are covered, which may benefit to the people new in Big Data world.
PageRank is a basic metric used to measure the importance of web pages. It is an iterative method... more PageRank is a basic metric used to measure the importance of web pages. It is an iterative method, and computed by Power method. Power method converges too slowly for the true structure of web. This issues encourages to find out some other iterative methods to accelerate PageRank computation. In this paper, we present a novel method that is combination of Quadratic Extrapolation and Gauss-Seidel called QE-GSeidel method which accelerate the convergence of PageRank method. From numerical experiment we show that QE-GSeidel method takes almost 60-65% less number of iteration and 25-30% less CPU time than Power, and takes 50-55% less number of iteration and 15-20% less CPU time than Gauss-Seidel in PageRank computation.
International Journal of Computer Applications, 2010
Association Rule Mining from transaction-oriented databases is one of the important process that ... more Association Rule Mining from transaction-oriented databases is one of the important process that finds relation between items and plays important role in decision making. Parallel algorithms are required because of large size of the database to be mined. Most of the algorithms designed were for homogeneous system uses static load balancing technique which is far from reality. A parallel algorithm for heterogeneous system is regarded as one of the most promising platforms for association rule mining. In this paper we propose a simple parallel algorithm for association rule mining on heterogeneous system with dynamic load balancing based on Par-Maxclique algorithm. We maintain one linked list at the scheduler end that keep track of load value of every processor and each processor is having a job queue associated with it which is served in First come first basis. On the basis of load value scheduler directs the migration of task from heavy loaded to least loaded processor in the cluster during the execution and thus balances load dynamically in a cluster.
2009 International Conference on Computer and Automation Engineering, 2009
... Algorithms Rakhi Garg, PK Mishra Department Of Computer Science, Banaras Hindu University, Va... more ... Algorithms Rakhi Garg, PK Mishra Department Of Computer Science, Banaras Hindu University, Varanasi, India e-mail: [email protected] ,[email protected] ... A. APRIORI, Sequential ARM algorithm -Proposed by Rakesh Agrawal and J. Shafer [14]. ...
Association Rule Mining (ARM) is one of the well know and most researched technique of data minin... more Association Rule Mining (ARM) is one of the well know and most researched technique of data mining. There are so many ARM algorithms have been designed that their counting is a large number. In this paper we have surveyed the various ARM algorithms in four computing environments. The considered computing environments are sequential computing, parallel and distributed computing, grid computing and cloud computing. With the emergence of new computing paradigm, ARM algorithms have been designed by many researchers to improve the efficiency by utilizing the new paradigm. This paper represents the journey of ARM algorithms started from sequential algorithms, and through parallel and distributed, and grid based algorithms to the current state-of-the-art, along with the motives for adopting new machinery.
Mining frequent itemsets from massive datasets is always being a most important problem of data m... more Mining frequent itemsets from massive datasets is always being a most important problem of data mining. Apriori is the most popular and simplest algorithm for frequent itemset mining. To enhance the efficiency and scalability of Apriori, a number of algorithms have been proposed addressing the design of efficient data structures, minimizing database scan and parallel and distributed processing. MapReduce is the emerging parallel and distributed technology to process big datasets on Hadoop Cluster. To mine big datasets it is essential to re-design the data mining algorithm on this new paradigm. In this paper, we implement three variations of Apriori algorithm using data structures hash tree, trie and hash table trie i.e. trie with hash technique on MapReduce paradigm. We emphasize and investigate the significance of these three data structures for Apriori algorithm on Hadoop cluster, which has not been given attention yet. Experiments are carried out on both real life and synthetic d...
Due to huge, unstructured and scattered amount of data available on web, it is very tough for use... more Due to huge, unstructured and scattered amount of data available on web, it is very tough for users to get relevant information in less time. To achieve this, improvement in design of web site, personalization of contents, prefetching and caching activities are done according to user’s behavior analysis. User’s activities can be captured into a special file called log file. There are various types of log: Server log, Proxy server log, Client/Browser log. These log files are used by web usage mining to analyze and discover useful patterns. The process of web usage mining involves three interdependent steps: Data preprocessing, Pattern discovery and Pattern analysis. Among these steps, Data preprocessing plays a vital role because of unstructured, redundant and noisy nature of log data. To improve later phases of web usage mining like Pattern discovery and Pattern analysis several data preprocessing techniques such as Data Cleaning, User Identification, Session Identification, Path Co...
Data cleaning plays an important role in Web usage mining process as it reduces the complexity of... more Data cleaning plays an important role in Web usage mining process as it reduces the complexity of later phases of KDD (Knowledge Discovery in Databases) process. It incorporates the removal of those log entries and fields that are not relevant for the purpose of Web usage mining. Generally, Data cleaning includes the removal of unsuccessful requests, irrelevant files’ requests, inappropriate access methods’ requests, and robots’ requests. With the enormous increase of log data, traditional Data cleaning algorithm has become highly data intensive task. In the last few years, MapReduce has emerged as most widely used parallel programming framework to compute data intensive application on a cluster of nodes. In this article, we have proposed a MapReduce-based Parallel Data cleaning (PDC) algorithm that considers all aspect of the traditional Data cleaning algorithm. There are two modifications to exiting Data cleaning algorithm i.e. first is an efficient robot detection approach, and o...
Association Rule Mining (ARM) is one of the well know and most researched technique of data minin... more Association Rule Mining (ARM) is one of the well know and most researched technique of data mining. There are so many ARM algorithms have been designed that their counting is a large number. In this paper we have surveyed the various ARM algorithms in four computing environments. The considered computing environments are sequential computing, parallel and distributed computing, grid computing and cloud computing. With the emergence of new computing paradigm, ARM algorithms have been designed by many researchers to improve the efficiency by utilizing the new paradigm. This paper represents the journey of ARM algorithms started from sequential algorithms, and through parallel and distributed, and grid based algorithms to the current state-of-the-art, along with the motives for adopting new machinery.
International Journal of Advanced Research in Computer Science, 2017
Due to proliferation of Web robots, it is becoming important to detect robots on commercial and e... more Due to proliferation of Web robots, it is becoming important to detect robots on commercial and educational websites. Web robots not only consumes a significant proportion of network bandwidth but also a threat for privacy and security. There are various techniques to detect robot from server log i.e. checking robots.txt access, matching keywords in User agent field, calculating browsing speed etc. In this paper, we have implemented and compared theabove three robot detection approaches on real Web server log. Moreover, we have also applied combined approach to detect robots and found that combined approach works better than other three approaches.
Association rule mining is a time consuming process due to involving both data intensive and comp... more Association rule mining is a time consuming process due to involving both data intensive and computation intensive nature. In order to mine large volume of data and to enhance the scalability and performance of existing sequential association rule mining algorithms, parallel and distributed algorithms are developed. These traditional parallel and distributed algorithms are based on homogeneous platform and are not lucrative for heterogeneous platform such as grid and cloud. This requires design of new algorithms which address the issues of good data set partition and distribution, load balancing strategy, optimization of communication and synchronization technique among processors in such heterogeneous system. Grid and cloud are the emerging platform for distributed data processing and various association rule mining algorithms have been proposed on such platforms. This survey article integrates the brief architectural aspect of distributed system, various recent approaches of grid ...
72 Abstract— Among various data mining task, association rule mining (ARM) is the major technique... more 72 Abstract— Among various data mining task, association rule mining (ARM) is the major technique which is widely used in retail marketing, bioinformatics, website navigation analysis etc. It finds correlations among items in a given data sets and establishes an association between two non overlapping sets of frequently occurring values in a database. Various sequential and parallel ARM algorithms have been developed that differs in data layout, search technique, data structure, the number of database scans used and the system on which it is developed i.e. homogeneous or heterogeneous systems. This paper mainly put emphasis in the need of a candidate based ARM algorithm for heterogeneous PC cluster that reduces the database scans and time complexity. It also describe the design and functioning of the heterogeneous PC cluster i.e. MPICH2 and the data decomposition technique applied for candidate itemsets generation that plays important role in balancing workload as well as enhancing ...
Data preprocessing is an essential task to prepare suitable target datasets to apply statistical ... more Data preprocessing is an essential task to prepare suitable target datasets to apply statistical and data mining algorithms. It has become one of the complex segment in Web usage mining due to the ...
Association rule mining is one of the major technique of data mining, involves finding of frequen... more Association rule mining is one of the major technique of data mining, involves finding of frequent itemsets with minimum support and generating association rule among them with minimum confidence. The task of finding all frequent itemsets for a large datasets requires a lot of computation which can be minimized by exploiting parallelism to the sequential algorithms. In this paper, we provide the preliminaries of basic concepts about association rule mining, different sequential association rule mining algorithms on different hardware platforms and also focus on the challenges in exploiting parallelism to these algorithms. We also discusses up to what extent these challenges e.g. load balancing, efficient memory usage, minimization of communication cost among processors, efficient data and task decomposition etc. are congregate by a given parallel association rule mining algorithm and classifies them accordingly. Although, this survey cannot be complete review of all algorithms, but ...
PageRank is one of the basic metric used in web search technology to rank the web pages. It uses ... more PageRank is one of the basic metric used in web search technology to rank the web pages. It uses power method to compute principal eigenvector of the web matrix of several billion nodes. PageRank method incorporates a parameter α called damping factor that plays a major role in PageRank computation. In this study, we have observed experimentally the efficiency of various iterative methods on hyperlink graph for different value of α. We conclude from experiment that Power method is effective and more competitive for the well condition problem i.e. small value of α. However, for α → 1 Power method becomes more complex, and other methods such as Aitken-Power, SOR, and Gauss-Seidel are more efficient than it in respect of CPU time as well as the number of iteration needed for convergence.
Big Data is an excessive amount of imprecise data in variety of formats generated from variety of... more Big Data is an excessive amount of imprecise data in variety of formats generated from variety of sources with rapid speed. It is most buzzed terms among researcher, industry and academia. Big Data is not only limited to data perspective but it has been emerged as a stream that includes associated technologies, tools and real word applications. The objective of this paper is to provide a simple, comprehensive and brief introduction of Big Data to the beginners in subject. In this paper, we provide an overview of Hadoop and its subprojects and a brief review of various developed technologies for Big Data. We also discuss some recent trends and eminent applications in Big Data. Although this paper does not touch each and every dimension of Big Data as it is not possible to make it in a single paper but essential aspects are covered, which may benefit to the people new in Big Data world.
Uploads
Papers by Rakhi Garg