Bayes network classification

Question

I'm on the process to learn Bayes network for classification on matlab, and I'm stuck on a simple (I think) step:

So for a naive bayes classifier like for the iris data set, the class is on the top node like this:

        class

        /  \ 
feature1    feature2

So that is ok I get it why the class is the cause of the features, that is ok, I get why the class by itself has a prior.

But in the case of a non-naive bayes network, like this:

cause1      cause2
     |  \        /
     |  consequence
     |    /
     class

In that situation that is no naive, how will the prior go, how do I set it up? How do I get the classification from this? Thanks (:

PS: I'm looking at BNT for a net.

Answer 1

Assuming all variables you mention are categorical and the edge directions are from up to down:

Priors:

In the first Naive Bayes example, the conditional probability table (CPT) of 'class' consists solely of its prior distribution because it is a root node, i.e. does not have any parents. If 'class' can take on 2 states (e.g. black and white), its CPT will consist of 2 values.

In the second Bayesian Network (BN) example, the CPT of 'class' is dependent on 'cause1' and 'consequence'. Lets say 'consequence' has 3 states, 'cause1' has 4 states and as before 'class' has 2 states. In this case, the CPT of 'class' would contain 3*4*2 values. When you are learning this CPT, you can incorporate your prior beliefs as a dirichlet distribution (if all variables are categorical). For an example of how to incorporate your prior beliefs into a Maximum Likelihood Estimation process, have a look at these excellent lecture slides.

Inference: (or what you refer to as 'classification')

As per carrying out the classification, in Example 1, you can make use of the Bayes Rule to calculate P('class' = white) and P('class' = black). In the second (BN) example, you would have to use a belief propagation or variable elimination or junction tree algorithm to update the posterior probabilities of the 'class' node based on your observed nodes.

There is a straight-forward BNT example on how to accomplish this here. Plus, BNT toolbox comes with brief 'inference' examples that use the junction tree function, which you can find under the .../examples folder.

Finally -some people may disagree but- as far as BNs go, I would advise against strictly interpreting A -> B as "A causes B" since the causality aspect of BNs, especially in the field of structure learning, is open to much debate.

I hope this helps.