A Hybrid Trainable Rule-based System

Neurules are a kind of hybrid rules integrating neurocomputing and production rules. Each neurule is represented as an adaline unit. Thus, the corresponding rule base consists of a number of autonomous adaline units (neurules). Due to this fact, a modular and natural rule base is constructed, in contrast to existing connectionist rule bases. In this paper, we present a method for generating neurules from empirical data. We overcome the difficulty of the adaline unit to classify non-separable training examples by introducing the notion of 'closeness' between training examples and splitting each training set into subsets of 'close' examples.

International Journal on Artificial Intelligence Tools, 2001

Neurules are a kind of hybrid rules integrating neurocomputing and production rules. Each neurule is represented as an adaline unit. Thus, the corresponding neurule base consists of a number of autonomous adaline units (neurules). Due to this fact, a modular and natural knowledge base is constructed, in contrast to existing connectionist knowledge bases. In this paper, we present a method for generating neurules from empirical data. To overcome the difficulty of the adaline unit to classify non-separable training examples, the notion of 'closeness' between training examples is introduced. In case of a training failure, two subsets of 'close' examples are produced from the initial training set and a copy of the neurule for each subset is trained. Failure of training any copy, leads to production of further subsets as far as success is achieved.

Expert Systems with Applications, 1991

Rule-based expert systems either develop out of the direct involvement of a concerned expert or through the enormous efforts of intermediaries called knowledge engineers . In either case, knowledge engineering tools are inadequate in many ways to support the complex problem of expert system building. This article describes a set of experiments with adaptive neural networks which explore two types of learning, deductive and inductive, in the context of a rule-based, deterministic parser of Natural Language. Rule-based processing of Language is an important and complex domain. Experiences gained in this domain generalize to other rule-based domains. We report on those experiences and draw some general conclusions that are relevant to knowledge engineering activities and maintenance of rule-based systems.

IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339)

Local basis function networks are a useful category of classifiers, with known variations developed in neural networks, machine learning and statistics communities. The localized range of activation of the hidden units have many similarities with rule-based representations. Neurofuzzy systems are a common example of an framework that explicitly integrates these approaches. Following this concept, we study alternatives for the development of rule-neural hybrid systems with the purpose of inducing robust and interpretable classifiers. Local fitting of parameters is done by a gradient descent optimization that modifies the covering produced by a rule induction algorithm. Two tasks are accomplished: how to select a small number of rules and how to improve precision. Accuracy is not the only target: indeed, the use of this architecture is better suited when one wants to achieve a good compromise between classification performance and simplicity.

Rule-extraction from trained neural networks has previously been used to generate propositional rule-sets. The extraction of "generic" rules or objects from trained feedforward networks is clearly desirable and sufficient for many applications. We present several approaches to generate a knowledge base that includes rules, facts and a is-a hierarchy that enables the greater explanatory capabilities by allowing the user interaction. The approaches are (1) construct two feedforward neural networks by cascade correlation algorithm [Fahlman & Lebiere, 1991] and tower algorithm ), extracts rules at the level of individual hidden and output units of both the networks by use of the decompositional rule-extraction method "LAP"; (2) cascade correlation and tower algorithm to train two different feedforward neural network, extracts rules that map inputs directly into outputs to generate the examples for each learning algorithm by the use of the pedagogical rule-extraction method "RuleVI" (3) constrained error back propagation to train a feedforward neural network, extract rules at the level of individual hidden and output units by use of the decompositional rule-extraction method "RULEX"; and use of the extracted symbolic rules to generate a connectionist knowledge base. Then the performance is demonstrated by a number of real-world applications.

Neural Processing Letters, 1998

Three neural-based methods for extraction of logical rules from data are presented. These methods facilitate conversion of graded response neural networks into networks performing logical functions. MLP2LN method tries to convert a standard MLP into a network performing logical operations (LN). C-MLP2LN is a constructive algorithm creating such MLP networks. Logical interpretation is assured by adding constraints to the cost function, forcing the weights to ±1 or 0. Skeletal networks emerge ensuring that a minimal number of logical rules are found. In both methods rules covering many training examples are generated before more specific rules covering exceptions. The third method, FSM2LN, is based on the probability density estimation. Several examples of performance of these methods are presented.

Artificial Intelligence in Engineering, 1993

This paper presents an improved version of a simple rule induction algorithm known as RULES ('RULe Extraction System'). Compared to RULES, the new algorithm generally is faster as it requires fewer rule searching operations in its induction process. Furthermore, it allows the user to specify the number of rules to be extracted, is able to deal with incomplete examples and can handle attributes with numerical as well as nominal values. The algorithm has been tested on several applications. Two of these applications, including the identification of a dynamic system, are described in the paper. The results obtained have demonstrated the strong performance of the algorithm.

2010

This paper describes an efficient rule generation algorithm, called rule generation from artificial neural networks (RGANN) to generate symbolic rules from ANNs. Classification rules are sought in many areas from automatic knowledge acquisition to data mining and ANN rule extraction. This is because classification rules possess some attractive features. They are explicit, understandable and verifiable by domain experts, and can be modified, extended and passed on as modular knowledge. A standard three-layer feedforward ANN is the basis of the algorithm. A four-phase training algorithm is proposed for backpropagation learning. Comparing them to the symbolic rules generated by other methods supports explicitness of the generated rules. Generated rules are comparable with other methods in terms of number of rules, average number of conditions for a rule, and predictive accuracy. Extensive experimental studies on several benchmarks classification problems, including breast cancer, wine, season, golf-playing, and lenses classification demonstrate the effectiveness of the proposed approach with good generalization ability.

This paper describes an efficient algorithm REx for generating symbolic rules from artificial neural network (ANN). Classification rules are sought in many areas from automatic knowledge acquisition to data mining and ANN rule extraction. This is because classification rules possess some attractive features. They are explicit, understandable and verifiable by domain experts, and can be modified, extended and passed on as modular knowledge. REx exploits the first order information in the data and finds shortest sufficient conditions for a rule of a class that can differentiate it from patterns of other classes. It can generate concise and perfect rules in the sense that the error rate of the rules is not worse than the inconsistency rate found in the original data. An important feature of rule extraction algorithm, REx, is its recursive nature. They are concise, comprehensible, order insensitive and do not involve any weight values. Extensive experimental studies on several benchmark...

1992

We describe a neural network, called RuleNet, that learns explicit, symbolic condition-action rules in a formal string manipulation domain. RuleNet discovers functional categories over elements of the domain, and, at various points during learning, extracts rules that operate on these categories. The rules are then injected back into RuleNet and training continues, in a process called iterative projection. By incorporating rules in this way, RuleNet exhibits enhanced learning and generalization performance over alternative neural net approaches. By integrating symbolic rule learning and subsymbolic category learning, RuleNet has capabilities that go beyond a purely symbolic system. We show how this architecture can be applied to the problem of case-role assignment in natural language processing, yielding a novel rule-based solution.