A parallel learning algorithm for Bayesian inference networks

2002

Most existing algorithms for structural learning of Bayesian networks are suitable for constructing small-sized networks which consist of several tens of nodes. In this paper, we present a novel approach to the efficient and relatively-precise induction of large-scale Bayesian networks with up to several hundreds of nodes. The approach is based on the concept of Markov blanket and makes use of the divide-and-conquer principle. The proposed method has been evaluated on two benchmark datasets and a real-life DNA microarray data, demonstrating the ability to learn the large-scale Bayesian network structure efficiently.

Sensors, 2019

Discovering the Bayesian network (BN) structure from big datasets containing rich causal relationships is becoming increasingly valuable for modeling and reasoning under uncertainties in many areas with big data gathered from sensors due to high volume and fast veracity. Most of the current BN structure learning algorithms have shortcomings facing big data. First, learning a BN structure from the entire big dataset is an expensive task which often ends in failure due to memory constraints. Second, it is quite difficult to select a learner from numerous BN structure learning algorithms to consistently achieve good learning accuracy. Lastly, there is a lack of an intelligent method that merges separately learned BN structures into a well structured BN network. To address these shortcomings, we introduce a novel parallel learning approach called PEnBayes (Parallel Ensemble-based Bayesian network learning). PEnBayes starts with an adaptive data preprocessing phase that calculates the Appropriate Learning Size and intelligently divides a big dataset for fast distributed local structure learning. Then, PEnBayes learns a collection of local BN Structures in parallel using a two-layered weighted adjacent matrix-based structure ensemble method. Lastly, PEnBayes merges the local BN Structures into a global network structure using the structure ensemble method at the global layer. For the experiment, we generate big data sets by simulating sensor data from patient monitoring, transportation, and disease diagnosis domains. The Experimental results show that PEnBayes achieves a significantly improved execution performance with more consistent and stable results compared with three baseline learning algorithms.

2012

Under the Supervision of Professor Istvan Lauko Statistics from the National Cancer Institute indicate that 1 in 8 women will develop Breast cancer in their lifetime. Researchers have developed numerous statistical models to predict breast cancer risk however physicians are hesitant to use these models because of disparities in the predictions they produce. In an effort to reduce these disparities, we use Bayesian networks to capture the joint distribution of risk factors, and simulate artificial patient populations (clinical avatars) for interrogating the existing risk prediction models. The challenge in this effort has been to produce a Bayesian network whose dependencies agree with literature and are good estimates of the joint distribution of risk factors. In this work, we propose a methodology for learning Bayesian networks that uses prior knowledge to guide a collection of search algorithms in identifying an optimum structure. Using data from the breast cancer surveillance consortium we have shown that our methodology produces a Bayesian network with consistent dependencies and a better estimate of the distribution of risk factors compared with existing methods.

Machine Learning, 1995

We describe algorithms for learning Bayesian networks from a combination of user knowledge and statistical data. The algorithms have two components: a scoring metric and a search procedure. The scoring metric takes a network structure, statistical data, and a user's prior knowledge, and returns a score proportional to the posterior probability of the network structure given the data. The search procedure generates networks for evaluation by the scoring metric. Our contributions are threefold. First, we identify two important properties of metrics, which we call score equivalence and parameter modu. iarity. These properties have been mostly ignored, but when combined, greatly simplify the encoding of a user's prior knowledge. In particular, a user can express his knowledge--for the most part--as a single prior Bayesian network for the domain. Second, we describe greedy hill-climbing and annealing search algorithms to be used in conjunction with scoring metrics. In the spec.is] case where each node has at most one parent, we show that heuristic search can be replaced with a polynomial algorithm to identify the networks with the highest score. Third, we describe a methodology for evaluating Bayesian-network learning algorithms. We apply this approach to a comparison of our metrics and search procedures.

The Journal of Machine …, 2004

In this paper, we provide new complexity results for algorithms that learn discrete-variable Bayesian networks from data. Our results apply whenever the learning algorithm uses a scoring criterion that favors the simplest structure for which the model is able to represent the ...

Information Processing and Management of Uncertainty in Knowledge-Based Systems, 2020

This paper considers the problem of learning a generalized credal network (a set of Bayesian networks) from a dataset. It is based on using the BDEu score and computes all the networks with score above a predetermined factor of the optimal one. To avoid the problem of determining the equivalent sample size (ESS), the approach also considers the possibility of an undetermined ESS. Even if the final result is a set of Bayesian networks, the paper also studies the problem of selecting a single network with some alternative procedures. Finally, some preliminary experiments are carried out with three small networks.

Knowledge Engineering Review, 1999

Bayesian Networks (BNs) model problems that involve uncertainty. A BN is a directed graph, whose nodes are the uncertain variables and whose edges are the causal or influential links between the variables. Associated with each node is a set of conditional probability functions that model the uncertain relationship between the node and its parents. The benefits of using BNs to

2017

We have already proposed a constraint-based learning Bayesian network method using Bayes factor. Since a conditional independence test using Bayes factor has consistency, the learning method improves the learning accuracy of the traditional constraint-based learning methods. Additionally, the method is expected to learn larger network structures than the traditional methods do because it greatly improves computational efficiency. However, its expected benefits have not been demonstrated empirically. This report describes some experiments related to the learning of large network structures. Results show that the proposed method can learn surprisingly huge networks with thousands of variables.

1998

A method for improving search-based inference techniques in Bayesian networks by obtaining a prior estimation of the error is presented. The method is based on a recently introduced algorithm for calculating the contribution of a given set of instantiations to the total probability mass. If a certain accuracy for the solution is desired, the method provides us with the number of replications (i.e., the sample size) needed for obtaining the approximated values with the desired accuracy. In addition to providing a prior stopping rule, the method substantially reduces the structure of the search tree and, hence, the computer time required for the process. Important savings are obtained in the case of Bayesian networks with extreme probabilities, as it is shown with the examples reported in the paper. As an example of several possible applications of the method, the problem of finding a maximal posteriori (MAP) instantiation of the Bayesian network variables, given a partial value assignment as an initial constraint, is presented.

2003

In this paper we describe how to learn Bayesian networks from a summary of complete data in the form of a dependency network rather than from data directly. This method allows us to gain the advantages of both representations: scalable algorithms for learning dependency networks and convenient inference with Bayesian networks. Our approach is to use a dependency network as an "oracle" for the statistics needed to learn a Bayesian network. We show that the general problem is NP-hard and develop a greedy search algorithm. We conduct a preliminary experimental evaluation and find that the prediction accuracy of the Bayesian networks constructed from our algorithm almost equals that of Bayesian networks learned directly from the data.

2017

Probabilistic graphical models are an attractive approach used for modelling complex systems, as the nature of the network allows uncertainty in the system to be accounted for. In particular, Bayesian networks, and their temporal extension, Dynamic Bayesian networks, are investigated. Difficulties often arise in the learning procedure, as the computational complexity of the network increases exponentially with each new variable. This can lead to a restriction on the number of variables to be included in the network and results in a network that is not truly representative of the system. This research wishes to demonstrate how these methods can benefit from using advanced computer technologies for implementing various learning algorithms, resulting in faster computation and the ability to handle larger data sets with an increased number of variables.

2004

In this paper, we propose two modifications to the original Minimum Description Length (MDL) score for learning of Bayesian networks. The first modification is that the description of network structure is proved to be unnecessary and can be omitted in the total MDL score. The second modification consists in reducing the description length of conditional probability table (CPT). In particular, if a specific variable is fully deterministic given its parents, i.e., the variable will take a certain value with probability one for some configurations of its parents, we show that only the configurations with probability one need to be retained in the CPT of the variable in the MDL score during the learning process of Bayesian networks. We name the MDL score with the above two modifications the Improved MDL score or IMDL score for short. The experimental results of classic Bayesian networks, such as ALARM and ASIA , show that the same search algorithm with the IMDL score can identify more reasonable and accurate models than those obtained with the original MDL score.

2021

Proceedings of the ... AAAI Conference on Artificial Intelligence, 2021

The creation of Bayesian networks often requires the specification of a large number of parameters, making it highly desirable to be able to learn these parameters from historical data. In many cases, such data has uncertainty associated with it, including cases in which this data comes from unstructured analysis or from sensors. When creating diagnosis networks, for example, unstructured analysis algorithms can be run on the historical text descriptions or images of previous cases so as to extract data for learning Bayesian network parameters, but such derived data has inherent uncertainty associated with it due to the nature of such algorithms. Because of the inability of current Bayesian network parameter learning algorithms to incorporate such uncertainty, common approaches either ignore this uncertainty, thus reducing the resulting accuracy, or completely disregard such data. We present an approach for learning Bayesian network parameters that explicitly incorporates such uncertainty, and which is a natural extension of the Bayesian network formalism. We present a generalization of the Expectation Maximization parameter learning algorithm that enables it to handle any historical data with likelihoodevidence-based uncertainty, as well as an empirical validation demonstrating the improved accuracy and convergence enabled by our approach. We also prove that our extended algorithm maintains the convergence and correctness properties of the original EM algorithm, while explicitly incorporating data uncertainty in the learning process.

Progress in Artificial Intelligence, 2015

One of the main research topics in machine learning nowadays is the improvement of the inference and learning processes in probabilistic graphical models. Traditionally, inference and learning have been treated separately, but given that the structure of the model conditions the inference complexity, most learning methods will sometimes produce inefficient inference models. In this paper we propose a framework for learning low inference complexity Bayesian networks. For that, we use a representation of the network factorization that allows efficiently evaluating an upper bound in the inference complexity of each model during the learning process. Experimental results show that the proposed methods obtain tractable models that improve the accuracy of the predictions provided by approximate inference in models obtained with a well-known Bayesian network learner.