What is the relationship between bayesian and neural networks?

Question

I'm looking for computationally heavy tasks to implement with CUDA and wonder if neural networks or bayesian networks might apply. This is not my question, though, but rather what the relation between the two network types is. They seem very related, especially if you look at bayesian networks with a learning capability (which the article on wikipedia mentions). At a glance, bayesian networks look at bit like a specific type of neural networks. Can anyone sum up their relationship, and if there is any connection beyond the apparent similarity?

Answer 1

Bayesian networks represent independence (and dependence) relationships between variables. Thus, the links represent conditional relationships in the probabilistic sense. Neural networks, generally speaking, have no such direct interpretation, and in fact the intermediate nodes of most neural networks are discovered features, instead of having any predicate associated with them in their own right.

Answer 2

It is reported that Bayesian networks are more resistant to the "overtraining" that is seen in some neural networks. In other words some neural networks become so "trained" to the observed measurements used in training that they aren't useful for the general cases.

Answer 3

Indeed they are. I see a bayesian network as a neural network applying the Baye's Theorem on large scale, but I don't remember details. I know where you can find them and I recommend this book for that.