Conditional Probabilities as Tensors?

Question

Is it proper to view conditional probabilities, such as the forms:

P(a|c)

P(a|c,d)

P(a, b|c, d)

...and so forth, as being tensors?

If so, does anyone know of a decent introductory text (online tutorial, workshop paper, book, etc) which develops tensors in that sense for computer scientists/machine learning practitioners?

I have found a number of papers, but those written at an introductory level are written for physicists, and those written for computer scientists are rather advanced.

Answer 1

This is definitely possible, although the tensor has of course certain additional structure (constraints).

If you consider the following conditional defined for a categorical response $Y$ on categorical predictors $X_i$:

$P(Y|X_1,\ldots,X_n)$

this correspond to a conditional probability tensor of size $d_0 \cdot d_1 \cdot \ldots \cdot d_n$. Here $Y$ (as a categorical response) takes $d_0$ levels.

Foremost the elements of the tensor have each to be nonnegative.

You can find papers on this, e.g. "Bayesian Conditional Tensor Factorizations for High-Dimensional Classification" by Yang and Dunson (pdf, arxiv).

Of course, if the random variables concerned are not categorical variables, but either continuous random variables or taking an infinite number of values (or both), the mapping to finite tensors won't be so trivial. You'll need something like an infinite tensor (as an extension of an infinite matrix).