The relationship between belief networks and relational databases is examined. Based on this anal... more The relationship between belief networks and relational databases is examined. Based on this analysis, a method to construct belief networks automatically from statistical rela tional data is proposed. A comparison be tween our method and other methods shows that our method has several advantages when generalization or prediction is deeded.
Parallel MCE reasoning and Boltzmann-Jeffrey machine networks
... evidence. As pointed out by Brown. afm one constraint set has been used * t sets ustdbefore m... more ... evidence. As pointed out by Brown. afm one constraint set has been used * t sets ustdbefore may to update the approximation, the ct" no longer be satisfied, and the last constraint stt dominates. ... Vi nuj cui nuk, and (3) NO BJMk(k<j) exists SO that Vi nu, = Vi nuk A UJ n n uj. ...
Automating the design of telecommunication distribution networks
This paper describes an automated system for the design of telecommunication distribution network... more This paper describes an automated system for the design of telecommunication distribution networks. The paper discusses models for the design process and shows how appropriate modelling and representation of the problem domain can simplify design. In particular, the paper shows how the decomposition of the problem in terms of the design components and the constraints on the design solutions can simplify the problem of generating new design structures. The embedding of the automated design process into a logical framework based on this decomposition is described, including the difficulties associated with representing all of the information required by the system that is taken for granted in manual design. The process of resource allocation to dimension the structure and provide a design solution is then described as well as how this process is optimised. Finally, the paper describes a technique for the optimisation of the resource allocation process.
Minimum Cross Entropy Reasoning in Recursive Causal Networks1 1This research is supported by a Commonwealth Postgraduate Research Award and a Sigma Data Research Award in Computing
Elsevier eBooks, 1990
A probabilistic method of reasoning under uncertainty is proposed based on the principle of Minim... more A probabilistic method of reasoning under uncertainty is proposed based on the principle of Minimum Cross Entropy (MCE) and concept of Recursive Causal Model (RCM). The dependency and correlations among the variables are described in a special language BNDL (Belief Networks Description Language). Beliefs are propagated among the clauses of the BNDL programs representing the underlying probabilistic distributions. BNDL interpreters in both Prolog and C has been developed and the performance of the method is compared with those of the others.
In this paper we describe a scheme for reasoning over causal netwotk.s in which known dependencie... more In this paper we describe a scheme for reasoning over causal netwotk.s in which known dependencies among variables can be included and multiple uncertain evidence can be present. Our scheme is based on the principle of Minimum Cross Entropy (MCE) [5, 9] and the concept of Recursive Causal Models (RCM) [4]. In this scheme: (1) We introduce a language, the Recursive Causal Networlcs Description Language (RCNDL), in which known dependencies among variables can be explicitly described. (2) We use known �ts about RCM to decompose the underlying probability space into subspaces, one for each RCNDL clause. Each subspace has its marginal distribution matching with the maximum likelihood estimation of the distribution of the whole space. (3) We propagate prior infonnation and beliefs among the clauses according to the MCE principle. For reasoning with multiple uncertain evidence, the constraint sets created by the evidence are used iteratively, and the principle of greatest gradient is used to order the constraint sets. An RCNDL interpreter has been developed and the scheme will be incorporated into J.L-Shell [14]. There has bee n much worlc on similar problems, this includes: Lemmer's Generalized Bayesian Updating Method [6). Using the Lagrange multiplier method. Lemmer derives Jeffrey's rule [3] by minimizing the cross entropy of the underlying distribution subject to one Component Marginal Distribution (CMD). This method produces an approximation of the object distribution when more than one CMD has to be considered simultaneously. When used with a tree of Local Event Groups (LEG) it can be quite efficient. Cheeseman's Method of Maximum Entropy [2]. Cheeseman uses the principle of Maximum Entropy (ME) to calculate the underlying distribution given some constraints by the traditional Lagrange multiplier method. To avoid the exponential explosion of the number of states as the number of variables in the space increases, Cheese man uses an efficient method to perform the relevant summ ations. Pearl's Method of Baytsian Networks (8]. Pearl •proposes an elegant and very efficient mechanism for propagating beliefs in parall el among the nodes in the causal networks which only needs local computation. However, it iS difficult to use this method in multi-connected causal networlcs. Spiegelhalter's Method of GraphicaURecursive Models in Contingency Tables [10]. According to the statistical theory of graphical/recursive models in contingency tables (16), this method decomposes the underlying space into subspaces and guarantees that the distributions of the subspaces This research is supported by a Commonwealth Postgraduaae Aw11'd and a Sigma Dara Research Award in Computing.
Reasoning under uncertainty is one of the most important challenges in expert systems and some ot... more Reasoning under uncertainty is one of the most important challenges in expert systems and some other branches of AI. Computational efficiency is a primary problem in implementing any practical system. In order to improve computational efficiency, several methods have been proposed to exploit the parallelism inherent in reasoning under uncertainty. However, some of these models can be used only in the case of singly connected networks, and one allows only one direction of reasoning. A parallel reasoning method based on the minimum cross entropy principle and the concept of recursive causal models is proposed to avoid the disadvantages of the methods.<<ETX>>
Annals of Mathematics and Artificial Intelligence, Mar 1, 1990
In this paper, the relationship between information and reasoning is investigated and a parallel ... more In this paper, the relationship between information and reasoning is investigated and a parallel reasoning method is proposed based on information theory, in particular the principle of minimum cross entropy. Some technical issues, such as multiple uncertain evidence, complicated constraints, small directed cycles and decomposition of underlying networks, are discussed. Some simple examples are also given to compare the method proposed here with other methods.
In this paper, optimum decomposition of belief networks is discussed. Some methods of decom posit... more In this paper, optimum decomposition of belief networks is discussed. Some methods of decom position are examined and a new method-the method of Minimum Total Number of States (MTNS)-is proposed. The problem of optimum belief network decomposition under our frame work, as under all the other frameworks, is shown to be NP-hard. According to the computational complexity analysis, an algorithm of belief net work decomposition is proposed in (Wen, 1989b) based on simulated annealing.
IEEE Transactions on Knowledge and Data Engineering, 1995
Expert critics have been built to critique human performance in various areas such as engineering... more Expert critics have been built to critique human performance in various areas such as engineering design, decision making, etc. We suggest that critics can also be useful in building and use of knowledge-based design systems (KBDSs). Knowledge engineers elicit knowledge from domain experts and build a knowledgebased design system. The system generates designs. The amount of knowledge the system possesses and the way it applies the knowledge directly in uence the performance of its designs. Therefore, critics are proposed to assist (1) acquiring su cient knowledge for constructing a desirable system, and (2) applying proper knowledge to generating designs. Methodologies of equipping a KBDS with critics are developed. Our practice in building and using a KBDS shows the applicability and capability of these critics.
Abstract The authors present an approach to optimizing the design of a knowledge-based design (KB... more Abstract The authors present an approach to optimizing the design of a knowledge-based design (KBD) system so that it produces optimal or near-optimal results. It is subjective in nature, however, as a KBD system's performance is often justified by the examination of its designs by experts. One alternative, as shown here, is to apply the simulated annealing technique (SA) in designing a KBD system. During the design of a KBD system, two designs by a KBD system and a SA system are independently generated. The solution generated ...
Abstract A knowledge-based design (KBD) system applies human expertise to create designs. The rig... more Abstract A knowledge-based design (KBD) system applies human expertise to create designs. The right choice of a particular set of heuristics for a given design is considered. The design problem is outlined, and a KB system that automates design in this domain is described. The methods used to deal with the ad hoc nature of such a system are discussed. It is demonstrated that the system generates better designs more often by choosing different sets of design rules in the light of varied situations. The problems, such as how to evaluate ...
La presente invention concerne un moteur et un systeme de recherche de donnees, notamment sur les... more La presente invention concerne un moteur et un systeme de recherche de donnees, notamment sur les pages Web d'Internet, comportant un analyseur de demande permettant de traiter une demande et attribuer des poids a chacun des termes de la demande et generer un vecteur de demande comprenant ces poids, et un reseau d'indices sensible au vecteur de demande pour produire au moins un indice pour les donnees en reponse a la demande. Le reseau d'indices est un reseau neurale a generation automatique construit en utilisant des exemples d'apprentissage deduits au moyen d'un extracteur de caracteristiques. L'extracteur de caracteristiques est utilise lors des phases de recherche et d'apprentissage. Un dispositif de regroupement est utilise pour grouper ces resultat de recherche.
In this paper, supervised learning for Self-Generating Neural Networks (SGNN) method, which was o... more In this paper, supervised learning for Self-Generating Neural Networks (SGNN) method, which was originally developed for the purpose of unsupervised learning, is discussed. An information analytical method is proposed to assign weights to attributes in the training examples if class information is available. This significantly improves the learning speed and the accuracy of the SGNN classifier. The performance of the supervised version of SGNN is analyzed and compared with those of other well-known supervised learning methods. 1 INTRODUCTION The SGNN (Self-Generating Neural Network) method proposed in [11] is an unsupervised learning method. It has been applied to different application areas such as image coding, diagnostic expert system and document /information retrieval system [10]. Although the SGNN method performs quite satisfactorily comparing with other unsupervised learning methods, it cannot compete with other supervised learning methods in some cases since the class inform...
this paper, we present a joint concept formation system, SGNN, that extends the previous work of ... more this paper, we present a joint concept formation system, SGNN, that extends the previous work of concept formation [Fisher, 1987; McKusick and Langley, 1991]. SGNN is able to generate either disjoint concept trees or acyclic directed concept graphs, according to the characteristics inherited in domain data. Furthermore, with certain controls applied to the number of winners at each concept layer, SGNN can also be used to only construct disjoint concept trees regardless of the regularity inherited in data. It is demonstrated 15
A probabilistic method of reasoning under uncertainty is proposed based on the principle of Minim... more A probabilistic method of reasoning under uncertainty is proposed based on the principle of Minimum Cross Entropy (MCE) and concept of Recursive Causal Model (RCM). The dependency and correlations among the variables are described in a special language BNDL (Belief Networks Description Language). Beliefs are propagated among the clauses of the BNDL programs representing the underlying probabilistic distributions. BNDL interpreters in both Prolog and C has been developed and the performance of the method is compared with those of the others.
A parallel distributed computational model for reasoning and learning is discussed based on a bel... more A parallel distributed computational model for reasoning and learning is discussed based on a belief network paradigm. Issues like reasoning and learning for the proposed model are discussed. Comparisons between our method and other methods are also given.
The most difficult task in probabilistic reasoning may be handling directed cycles in belief netw... more The most difficult task in probabilistic reasoning may be handling directed cycles in belief networks. To the best knowledge of this author, there is no serious discussion of this problem at all in the literature of probabilistic reasoning so far.
The relationship between belief networks and relational databases is examined. Based on this anal... more The relationship between belief networks and relational databases is examined. Based on this analysis, a method to construct belief networks automatically from statistical rela tional data is proposed. A comparison be tween our method and other methods shows that our method has several advantages when generalization or prediction is deeded.
Parallel MCE reasoning and Boltzmann-Jeffrey machine networks
... evidence. As pointed out by Brown. afm one constraint set has been used * t sets ustdbefore m... more ... evidence. As pointed out by Brown. afm one constraint set has been used * t sets ustdbefore may to update the approximation, the ct" no longer be satisfied, and the last constraint stt dominates. ... Vi nuj cui nuk, and (3) NO BJMk(k<j) exists SO that Vi nu, = Vi nuk A UJ n n uj. ...
Automating the design of telecommunication distribution networks
This paper describes an automated system for the design of telecommunication distribution network... more This paper describes an automated system for the design of telecommunication distribution networks. The paper discusses models for the design process and shows how appropriate modelling and representation of the problem domain can simplify design. In particular, the paper shows how the decomposition of the problem in terms of the design components and the constraints on the design solutions can simplify the problem of generating new design structures. The embedding of the automated design process into a logical framework based on this decomposition is described, including the difficulties associated with representing all of the information required by the system that is taken for granted in manual design. The process of resource allocation to dimension the structure and provide a design solution is then described as well as how this process is optimised. Finally, the paper describes a technique for the optimisation of the resource allocation process.
Minimum Cross Entropy Reasoning in Recursive Causal Networks1 1This research is supported by a Commonwealth Postgraduate Research Award and a Sigma Data Research Award in Computing
Elsevier eBooks, 1990
A probabilistic method of reasoning under uncertainty is proposed based on the principle of Minim... more A probabilistic method of reasoning under uncertainty is proposed based on the principle of Minimum Cross Entropy (MCE) and concept of Recursive Causal Model (RCM). The dependency and correlations among the variables are described in a special language BNDL (Belief Networks Description Language). Beliefs are propagated among the clauses of the BNDL programs representing the underlying probabilistic distributions. BNDL interpreters in both Prolog and C has been developed and the performance of the method is compared with those of the others.
In this paper we describe a scheme for reasoning over causal netwotk.s in which known dependencie... more In this paper we describe a scheme for reasoning over causal netwotk.s in which known dependencies among variables can be included and multiple uncertain evidence can be present. Our scheme is based on the principle of Minimum Cross Entropy (MCE) [5, 9] and the concept of Recursive Causal Models (RCM) [4]. In this scheme: (1) We introduce a language, the Recursive Causal Networlcs Description Language (RCNDL), in which known dependencies among variables can be explicitly described. (2) We use known �ts about RCM to decompose the underlying probability space into subspaces, one for each RCNDL clause. Each subspace has its marginal distribution matching with the maximum likelihood estimation of the distribution of the whole space. (3) We propagate prior infonnation and beliefs among the clauses according to the MCE principle. For reasoning with multiple uncertain evidence, the constraint sets created by the evidence are used iteratively, and the principle of greatest gradient is used to order the constraint sets. An RCNDL interpreter has been developed and the scheme will be incorporated into J.L-Shell [14]. There has bee n much worlc on similar problems, this includes: Lemmer's Generalized Bayesian Updating Method [6). Using the Lagrange multiplier method. Lemmer derives Jeffrey's rule [3] by minimizing the cross entropy of the underlying distribution subject to one Component Marginal Distribution (CMD). This method produces an approximation of the object distribution when more than one CMD has to be considered simultaneously. When used with a tree of Local Event Groups (LEG) it can be quite efficient. Cheeseman's Method of Maximum Entropy [2]. Cheeseman uses the principle of Maximum Entropy (ME) to calculate the underlying distribution given some constraints by the traditional Lagrange multiplier method. To avoid the exponential explosion of the number of states as the number of variables in the space increases, Cheese man uses an efficient method to perform the relevant summ ations. Pearl's Method of Baytsian Networks (8]. Pearl •proposes an elegant and very efficient mechanism for propagating beliefs in parall el among the nodes in the causal networks which only needs local computation. However, it iS difficult to use this method in multi-connected causal networlcs. Spiegelhalter's Method of GraphicaURecursive Models in Contingency Tables [10]. According to the statistical theory of graphical/recursive models in contingency tables (16), this method decomposes the underlying space into subspaces and guarantees that the distributions of the subspaces This research is supported by a Commonwealth Postgraduaae Aw11'd and a Sigma Dara Research Award in Computing.
Reasoning under uncertainty is one of the most important challenges in expert systems and some ot... more Reasoning under uncertainty is one of the most important challenges in expert systems and some other branches of AI. Computational efficiency is a primary problem in implementing any practical system. In order to improve computational efficiency, several methods have been proposed to exploit the parallelism inherent in reasoning under uncertainty. However, some of these models can be used only in the case of singly connected networks, and one allows only one direction of reasoning. A parallel reasoning method based on the minimum cross entropy principle and the concept of recursive causal models is proposed to avoid the disadvantages of the methods.<<ETX>>
Annals of Mathematics and Artificial Intelligence, Mar 1, 1990
In this paper, the relationship between information and reasoning is investigated and a parallel ... more In this paper, the relationship between information and reasoning is investigated and a parallel reasoning method is proposed based on information theory, in particular the principle of minimum cross entropy. Some technical issues, such as multiple uncertain evidence, complicated constraints, small directed cycles and decomposition of underlying networks, are discussed. Some simple examples are also given to compare the method proposed here with other methods.
In this paper, optimum decomposition of belief networks is discussed. Some methods of decom posit... more In this paper, optimum decomposition of belief networks is discussed. Some methods of decom position are examined and a new method-the method of Minimum Total Number of States (MTNS)-is proposed. The problem of optimum belief network decomposition under our frame work, as under all the other frameworks, is shown to be NP-hard. According to the computational complexity analysis, an algorithm of belief net work decomposition is proposed in (Wen, 1989b) based on simulated annealing.
IEEE Transactions on Knowledge and Data Engineering, 1995
Expert critics have been built to critique human performance in various areas such as engineering... more Expert critics have been built to critique human performance in various areas such as engineering design, decision making, etc. We suggest that critics can also be useful in building and use of knowledge-based design systems (KBDSs). Knowledge engineers elicit knowledge from domain experts and build a knowledgebased design system. The system generates designs. The amount of knowledge the system possesses and the way it applies the knowledge directly in uence the performance of its designs. Therefore, critics are proposed to assist (1) acquiring su cient knowledge for constructing a desirable system, and (2) applying proper knowledge to generating designs. Methodologies of equipping a KBDS with critics are developed. Our practice in building and using a KBDS shows the applicability and capability of these critics.
Abstract The authors present an approach to optimizing the design of a knowledge-based design (KB... more Abstract The authors present an approach to optimizing the design of a knowledge-based design (KBD) system so that it produces optimal or near-optimal results. It is subjective in nature, however, as a KBD system's performance is often justified by the examination of its designs by experts. One alternative, as shown here, is to apply the simulated annealing technique (SA) in designing a KBD system. During the design of a KBD system, two designs by a KBD system and a SA system are independently generated. The solution generated ...
Abstract A knowledge-based design (KBD) system applies human expertise to create designs. The rig... more Abstract A knowledge-based design (KBD) system applies human expertise to create designs. The right choice of a particular set of heuristics for a given design is considered. The design problem is outlined, and a KB system that automates design in this domain is described. The methods used to deal with the ad hoc nature of such a system are discussed. It is demonstrated that the system generates better designs more often by choosing different sets of design rules in the light of varied situations. The problems, such as how to evaluate ...
La presente invention concerne un moteur et un systeme de recherche de donnees, notamment sur les... more La presente invention concerne un moteur et un systeme de recherche de donnees, notamment sur les pages Web d'Internet, comportant un analyseur de demande permettant de traiter une demande et attribuer des poids a chacun des termes de la demande et generer un vecteur de demande comprenant ces poids, et un reseau d'indices sensible au vecteur de demande pour produire au moins un indice pour les donnees en reponse a la demande. Le reseau d'indices est un reseau neurale a generation automatique construit en utilisant des exemples d'apprentissage deduits au moyen d'un extracteur de caracteristiques. L'extracteur de caracteristiques est utilise lors des phases de recherche et d'apprentissage. Un dispositif de regroupement est utilise pour grouper ces resultat de recherche.
In this paper, supervised learning for Self-Generating Neural Networks (SGNN) method, which was o... more In this paper, supervised learning for Self-Generating Neural Networks (SGNN) method, which was originally developed for the purpose of unsupervised learning, is discussed. An information analytical method is proposed to assign weights to attributes in the training examples if class information is available. This significantly improves the learning speed and the accuracy of the SGNN classifier. The performance of the supervised version of SGNN is analyzed and compared with those of other well-known supervised learning methods. 1 INTRODUCTION The SGNN (Self-Generating Neural Network) method proposed in [11] is an unsupervised learning method. It has been applied to different application areas such as image coding, diagnostic expert system and document /information retrieval system [10]. Although the SGNN method performs quite satisfactorily comparing with other unsupervised learning methods, it cannot compete with other supervised learning methods in some cases since the class inform...
this paper, we present a joint concept formation system, SGNN, that extends the previous work of ... more this paper, we present a joint concept formation system, SGNN, that extends the previous work of concept formation [Fisher, 1987; McKusick and Langley, 1991]. SGNN is able to generate either disjoint concept trees or acyclic directed concept graphs, according to the characteristics inherited in domain data. Furthermore, with certain controls applied to the number of winners at each concept layer, SGNN can also be used to only construct disjoint concept trees regardless of the regularity inherited in data. It is demonstrated 15
A probabilistic method of reasoning under uncertainty is proposed based on the principle of Minim... more A probabilistic method of reasoning under uncertainty is proposed based on the principle of Minimum Cross Entropy (MCE) and concept of Recursive Causal Model (RCM). The dependency and correlations among the variables are described in a special language BNDL (Belief Networks Description Language). Beliefs are propagated among the clauses of the BNDL programs representing the underlying probabilistic distributions. BNDL interpreters in both Prolog and C has been developed and the performance of the method is compared with those of the others.
A parallel distributed computational model for reasoning and learning is discussed based on a bel... more A parallel distributed computational model for reasoning and learning is discussed based on a belief network paradigm. Issues like reasoning and learning for the proposed model are discussed. Comparisons between our method and other methods are also given.
The most difficult task in probabilistic reasoning may be handling directed cycles in belief netw... more The most difficult task in probabilistic reasoning may be handling directed cycles in belief networks. To the best knowledge of this author, there is no serious discussion of this problem at all in the literature of probabilistic reasoning so far.
Uploads
Papers by Wilson Wen