A Tutorial on Learning With Bayesian Networks
David Heckerman


Introduction:

A Bayesian network is a graphical model for probabilistic relationships among a set of
variables Over the last decade the Bayesian network has become a popular representation
for encoding uncertain expert knowledge in expert systems Heckerman et al a  More
recently researchers have developed methods for learning Bayesian networks from data The
techniques that have been developed are new and still evolving but they have been shown
to be remarkably eective for some data
analysis problems
In this paper we provide a tutorial on Bayesian networks and associated Bayesian
techniques for extracting and encoding knowledge from data There are numerous rep
resentations available for data analysis including rule bases decision trees and articial
neural networks and there are many techniques for data analysis such as density estima
tion classication regression and clustering So what do Bayesian networks and Bayesian
methods have to oer There are at least four answers
One Bayesian networks can readily handle incomplete data sets For example consider
a classication or regression problem where two of the explanatory or input variables are
strongly anti
correlated This correlation is not a problem for standard supervised learning
techniques provided all inputs are measured in every case When one of the inputs is not
observed however most models will produce an inaccurate prediction because they do not

encode the correlation between the input variables Bayesian networks oer a natural way
to encode such dependencies
Two Bayesian networks allow one to learn about causal relationships Learning about
causal relationships are important for at least two reasons The process is useful when we
are trying to gain understanding about a problem domain for example during exploratory
data analysis In addition knowledge of causal relationships allows us to make predictions
in the presence of interventions For example a marketing analyst may want to know
whether or not it is worthwhile to increase exposure of a particular advertisement in order
to increase the sales of a product To answer this question the analyst can determine
whether or not the advertisement is a cause for increased sales and to what degree The
use of Bayesian networks helps to answer such questions even when no experiment about
the eects of increased exposure is available
Three Bayesian networks in conjunction with Bayesian statistical techniques facilitate
the combination of domain knowledge and data Anyone who has performed a real
world
analysis knows the importance of prior or domain knowledge especially when data is scarce
or expensive The fact that some commercial systems ie expert systems  can be built from
prior knowledge alone is a testament to the power of prior knowledge Bayesian networks
have a causal semantics that makes the encoding of causal prior knowledge particularly
straightforward In addition Bayesian networks encode the strength of causal relationships
with probabilities Consequently prior knowledge and data can be combined with well
studied techniques from Bayesian statistics
Four Bayesian methods in conjunction with Bayesian networks and other types of mod
els oers an ecient and principled approach for avoiding the over tting of data As we
shall see there is no need to hold out some of the available data for testing Using the
Bayesian approach models can be smoothed in such a way that all available data can be
used for training
This tutorial is organized as follows In Section  we discuss the Bayesian interpretation
of probability and review methods from Bayesian statistics for combining prior knowledge
with data In Section  we describe Bayesian networks and discuss how they can be con
structed from prior knowledge alone In Section  we discuss algorithms for probabilistic
inference in a Bayesian network In Sections  and  we show how to learn the probabilities
in a xed Bayesian
network structure and describe techniques for handling incomplete data
including Monte
Carlo methods and the Gaussian approximation In Sections  through 
we show how to learn both the probabilities and structure of a Bayesian network Topics
discussed include methods for assessing priors for Bayesian
network structure and parame
ters and methods for avoiding the overtting of data including Monte
Carlo Laplace BIC

and MDL approximations In Sections  and  we describe the relationships between
Bayesian
network techniques and methods for supervised and unsupervised learning In
Section  we show how Bayesian networks facilitate the learning of causal relationships
In Section  we illustrate techniques discussed in the tutorial using a real
world case study
In Section  we give pointers to software and additional literature