Rule extraction from neural networks — A comparative study

2016

Classification and Rule extraction is an important application of Artificial Neural Network. To extract fewer rules from multilayer feed forward neural network has been a research area. The internal representation of the network is augmented by a distance term to extract fewer rules from the feed forward neural network and experimented on five datasets. Understanding affect of different factors of the dataset and network on extraction of a number of rules from the network can reveal important pieces of information which may help researchers to enhance the rule extraction process. This work investigates the internal behavior of neural network in rule extraction process on five different dataset. Keywords: Rule extraction, Feed Forward Neural Network, Hidden units, Activation value, Hidden neurons.

International journal of neural systems, 2001

The problem of rule extraction from neural networks is NP-hard. This work presents a new technique to extract "if-then-else" rules from ensembles of DIMLP neural networks. Rules are extracted in polynomial time with respect to the dimensionality of the problem, the number of examples, and the size of the resulting network. Further, the degree of matching between extracted rules and neural network responses is 100%. Ensembles of DIMLP networks were trained on four data sets in the public domain. Extracted rules were on average significantly more accurate than those extracted from C4.5 decision trees.

Classification is one of the data mining problems receiving great attention recently in the database community. This paper proposes a comparative study to highlight the significant difference between the C4.5 decision tree based algorithm and the neural network approach for classification and rule extraction. We compare the rules generated by the C4.5 with that generated by the RX (Neural Network based Algorithm). This is experimentally evaluated in different domains. The Experimental results demonstrate that rules extracted from neural networks are comparable with those of decision trees in terms of predictive accuracy, number of rules and average number of conditions for a rule.

International Journal of Bio-Inspired Computation, 2016

Classification is one of the important tasks of data mining and neural network is one of the best known tools for doing this task. Despite of producing high classification accuracy, the black box nature of neural network makes it useless for many applications which require transparency in its decision-making process. This drawback is overcome by extracting rules from neural network. Rule extraction makes neural network an alternative to other machine learning methods for handling classification problems by deriving an explanation of how each decision is made. Till now, many algorithms on rule extraction have been proposed but still research on this area is going on to find out more accurate and understandable rules. The proposed algorithm extracts rules from trained neural network for datasets with mixed mode attributes using pedagogical approach. The proposed algorithm uses classified and misclassified patterns to find out the data ranges of significant attributes in respective classes. The experimental results clearly show that the proposed algorithm produces accurate and understandable rules compared to existing algorithms.

The 4th European Conference on Principles and …, 2000

The inherent black-box nature of neural networks is an important drawback with respect to the problem of knowledge discovery from databases. In this work our aim is to extract rules from multi-layer perceptrons. This is a starting point to explain the basis of neural network solutions for knowledge discovery to experts of domain applications. Our approach consists in characterizing discriminant hyper-plane frontiers built by a special neural network model denoted to as Discretized Interpretable Multi Layer Perceptron (DIMLP). Rules are extracted in polynomial time with respect to the size of the problem and the size of the network. Further, the degree of matching between extracted rules and neural network responses is 100%. We apply DIMLP to 9 databases related to the medical diagnosis domain in which for some of them it gives better average predictive accuracy than standard multi-layer perceptrons and C4.5 decision trees. Finally, the quality of rules generated from DIMLP networks is compared to those related to decision trees.

Despite the highest classification accuracy in wide varieties of application areas, artificial neural network has one disadvantage. The way this Network comes to a decision is not easily comprehensible. The lack of explanation ability reduces the acceptability of neural network in data mining and decision system. This drawback is the reason why researchers have proposed many rule extraction algorithms to solve the problem. Recently, Deep Neural Network (DNN) is achieving a profound result over the standard neural network for classification and recognition problems. It is a hot machine learning area proven both useful and innovative. This paper has thoroughly reviewed various rule extraction algorithms, considering the classification scheme: decompositional, pedagogical, and eclectics. It also presents the evaluation of these algorithms based on the neural network structure with which the algorithm is intended to work. The main contribution of this review is to show that there is a limited study of rule extraction algorithm from DNN.

Neural Networks and Soft Computing, 2003

This paper presents our approach to the rule extraction problem from trained neural network. A method called REX is briefly described. REX acquires a set of fuzzy rules using an evolutionary algorithm. Evolutionary algorithm searches not only fuzzy rules, but also a description of fuzzy sets. The way of coding and evaluation process of an individual is presented. The method was tested using the following benchmark data sets: IRIS, WINE and Wisconsin Breast Cancer Diagnosis. On the basis of the experimental studies shown in this paper, we can conclude that rules obtained by REX can be easily understood by human-they include small number of premises, and their fidelity is very high. Obtained results are compared to other rule extraction methods.

Given paper is a review on existing decompositional rules extraction methods from artificial neural networks of several types: feed-forward network, radial basis functions network, second order reccurent network, generalized relevance learning vector quantization and finally support vector machine. Descriptions of all rules extraction methods are containing details on method itself, type of rules extracted, applicable problems and some test results.

Lecture Notes in Computer Science, 2001

The paper focuses on the problem of rule extraction from neural networks, with the aim of transforming the knowledge captured in a trained neural network into a familiar form for human user. The ultimate purpose for us is to develop human friendly shells for neural network based systems. In the first part of the paper it is presented an approach on extracting traditional crisp rules out of the neural networks, while the last part of the paper presents how to transform the neural network into a set of fuzzy rules using an interactive fuzzy operator. The rules are extracted from ordinary neural networks, which have not a structure that facilitate the rule extraction. The neural network trained with the well known Iris data set was considered as benchmark problem.

Pattern Recognition Letters, 1999

Search methods for rule extraction from neural networks work by ®nding those combinations of inputs that make the neuron active. By sorting the input weights to a neuron and ordering the weights suitably, it is possible to prune the search space. Based on this observation, we present an algorithm for rule extraction from feedforward neural networks with boolean inputs and analyze its properties.

… of the 8th Australian and New …, 2003

Rule extraction from neural networks often focusses on exact equivalence and is often tested on relatively small canonical examples. We apply genetic algorithms to the extract approximate rules from neural networks. The method is robust and works with large networks. We compare the results with rules obtained using state of the art decision tree methods and achieve superior performance to straight forward application of the WEKA implementation of the C5 algorithm, J48.PART.

One of the important discussions in data mining is extracting effective and useful rules from the great set of datasets. So, we should follow set of features that at first; are without any noise; secondly, having a little correlation with other features. In other words, we should use instances that are distinctive with other features. So, in this paper we present a combined approach to consider how factors such as distinct features and instances are useful for extracting the rules. In this approach we used a trained neural network to explore useful features, clustering to find out the best instances from dataset and finally we used artificial immune system for rules extraction. In order to evaluating of our introduced approach, we applied it on the UCI dataset of breast cancer diagnosis. Our experiments demonstrate that the combined proposed approach generates reliable rules and contributes more accuracy eventually; these results show the proposed method has %5.9 better accuracy relative to CART method.

Computing Research Repository, 2006

Artificial neural networks (ANNs) have been successfully applied to solve a variety of classification and function approximation problems. Although ANNs can generally predict better than decision trees for pattern classification problems, ANNs are often regarded as black boxes since their predictions cannot be explained clearly like those of decision trees. This paper presents a new algorithm, called rule extraction from

Nonlinear Analysis: Theory, Methods & …, 1997

New Generation Computing, 2018

Neural network is one of the best tools for data mining tasks due to its high accuracy. However, one of the drawbacks of neural network is its black box nature. This limitation makes neural network useless for many applications which require transparency in their decision-making process. Many algorithms have been proposed to overcome this drawback by extracting transparent rules from neural network, but still researchers are in search for algorithms that can generate more accurate and simple rules. Therefore, this paper proposes a rule extraction algorithm named Eclectic Rule Extraction from Neural Network Recursively (ERENNR), with the aim to generate simple and accurate rules. ERENNR algorithm extracts symbolic classification rules from a single-layer feed-forward neural network. The novelty of this algorithm lies in its procedure of analyzing the nodes of the network. It analyzes a hidden node based on data ranges of input attributes with respect to its output and analyzes an output node using logical combination of the outputs of hidden nodes with respect to output class. And finally it generates a rule set by proceeding in a backward direction starting from the output layer. For each rule in the set, it repeats the whole process of rule extraction if the rule satisfies certain criteria. The algorithm is validated with eleven benchmark datasets. Experimental results show that the generated rules are simple and accurate.