Scikit's Hidden Markov Model accepting observation probabilities that don't add up to 1

Question

I am trying to build a toy Hidden Markov Model with 2 states and 3 possible observations using the "MultinomialHMM" module, part of the scikit-learn library. My problem is that the module accepts (and generates predictions) even when the observation probabilities for a state add up to more than 1 or less than 1. example:

import numpy
from sklearn import hmm
startprob = numpy.array([0.5, 0.5])
transition_matrix = numpy.array([[0.5, 0.5], [0.5, 0.5]])
model = hmm.MultinomialHMM(2, startprob, transition_matrix)
model.emissionprob_ = numpy.array([[0, 0, 0.2], [0.6, 0.4, 0]])

Notice that the probabilities of the signals emitted by state 0 are [0,0,0.2] (which add up to 0.2). The module does not complain when asked to generate a sample of observations:

model.sample(10) 
(array([1, 0, 0, 0, 0, 2, 1, 0, 0, 0], dtype=int64), array([1, 1, 0, 1, 1, 0, 1, 0, 0, 0]))

I can also specify emission probailities that sum up to more than 1, and the model genenrates predictions with no complaints.

Is this desired behavior? Are the probabilities being normalized in someway? If so, how?

Answer 1

First of all, HMM is deprecated in sklearn. You need to check out https://github.com/hmmlearn/hmmlearn, which is Hidden Markov Models in Python, with scikit-learn like API

BTW, The problem you ask seems like a bug. When you set emissionprob_, the _set_emissionprob is called. This tries re-normalize by calling normalize(emissionprob):

if not np.alltrue(emissionprob):
    normalize(emissionprob)

However, this code has two problems:

1. dose not set axis properly.
2. not in-place even though the document says that.

So modified as

if not np.alltrue(emissionprob):
    normalize(emissionprob, 1) # added axis term

and

def normalize(A, axis=None):
    A += EPS
    Asum = A.sum(axis)
    if axis and A.ndim > 1:
        # Make sure we don't divide by zero.
        Asum[Asum == 0] = 1
        shape = list(A.shape)
        shape[axis] = 1
        Asum.shape = shape
    A /= Asum # this is true in-place, it was `return A / Asum`  <<= here