Rules Extraction Via Neural Networks Previously Trained

Informatik aktuell, 1994

Knowledge acquisition is a frequent bottleneck in artificial intelligence applications. Neural learning may offer a new perspective in this field. Using Self-Organising Neural Networks, as the Kohonen model, the inherent structures in high-dimensional input spaces are projected on a low dimensional space. The exploration of structures resp. classes is then possible applying the U-Matrix method for the visualisation of data. Since Neural Networks are not able to explain the obtained results, a machine learning algorithm sig* was developed to extract symbolic knowledge in form of rules out of subsymbolic data. Combining both approaches in hybrid system results in a powerful method to solve classification and diagnosis problems. Several applications have been used to test this method. Applications on processes with dynamic characteristics, such as chemical processes and avalanche forecasting show that an extension of this method from static to dynamic data is feasible.

2000

Artificial neural networks have been successfully applied to solve a variety of business applications involving classification and function approximation. In many such applications, it is desirable to extract knowledge from trained neural networks so that the users can gain a better understanding of the solution. Existing research works have focused primarily on extracting symbolic rules for classification problems with few methods devised for function approximation problems. In order to fill this gap, we propose an approach to extract rules from neural networks that have been trained to solve function approximation problems. The extracted rules divide the data samples into groups. For all samples within a group, a linear function of the relevant input attributes of the data approximates the network output. Experimental results show that the proposed approach generates rules that are more accurate than the existing methods based on decision trees and regression.

Given paper is a review on existing decompositional rules extraction methods from artificial neural networks of several types: feed-forward network, radial basis functions network, second order reccurent network, generalized relevance learning vector quantization and finally support vector machine. Descriptions of all rules extraction methods are containing details on method itself, type of rules extracted, applicable problems and some test results.

Nonlinear Analysis: Theory, Methods & …, 1997

Intelligent Information Processing and Web Mining, 2004

The presented paper describes a method of knowledge extraction that is based on analysis of the trained ANN's weights The method allows to determine the significance of particular inputs, to prove their synergy as well as to find some symbolic rules, that determine the direction of influence of particular inputs.

2005

A major drawback of artificial neural networks is their black-box character. Even when the trained network performs adequately, it is very difficult to understand its operation. In this paper, we use the mathematical equivalence between artificial neural networks and a specific fuzzy rule base to extract the knowledge embedded in the network. We demonstrate this using a benchmark problem: the recognition of digits produced by a LED device. The method provides a symbolic and comprehensible description of the knowledge learned by the network during its training.

International Conference on Pattern Recognition, Informatics and Medical Engineering (PRIME-2012), 2012

Though neural networks have achieved highest classification accuracy for many classification problems, the obtained results may not be interpretable as they are often considered as black box. To overcome this drawback researchers have developed many rule extraction algorithms. This paper has discussed on various rule extraction algorithms based on three different rule extraction approaches namely decompositional, pedagogical and eclectic. Also it evaluates the performance of those approaches by comparing different algorithms with these three approaches on three real datasets namely Wisconsin breast cancer, Pima Indian diabetes and Iris plants.

2000

Contrary to the common opinion neural networks may be used for knowledge extraction. Recently a new methodology of logical rule extraction, optimization and application of rule-based systems has been described. C-MLP2LN algorithm, based on constrained multilayer perceptron network, is described here in details and the dynamics of a transition from neural to logical system illustrated. The algorithm handles real-valued features, determining appropriate linguistic variables or membership functions as a part of the rule extraction process. Initial rules are optimized exploring the tradeoff between accuracy/simplicity at the rule extraction stage and between reliability of rules and rejection rate at the optimization stage. Gaussian uncertainties of measurements are assumed during application of crisp logical rules, leading to "soft trapezoidal" membership functions and allowing to optimize the linguistic variables using gradient procedures. Comments are made on application of neural networks to knowledge discovery in benchmark and in real life problems.

2006

Abstract. neural networks solving approximation problem. It is based on two hierarchical evolutionary algorithms with multiobjective Pareto optimisation. The lower level algorithm searches for rules that are optimised by the upper level algorithm. The conclusion of the ...

Information Technology and Management Science

Artificial neural networks are widely spread models that outperform more basic, but explainable machine learning models like classification decision tree. Although their lack of explainability severely limits their area of application. All mission critical areas or law regulated areas (like European GDPR) require model to be explained. Explainability allows model validation for correctness and lack of bias. Thus methods for knowledge extraction from artificial neural networks have gained attention and development efforts. Current paper addresses this problem and describes knowledge extraction methodology which can be applied to classification problems. It is based on previous research and allows knowledge to be extracted from trained fully connected feed-forward artificial neural network, from radial basis function neural network and from hyper-polytope based classifier in the form of binary classification decision tree, elliptical rules and If-Then rules.