Generating a Markov model from a matrix

Question

The definition may be wrong, so please correct me if that is so.. I need to generate a Markov model from a matrix of a following kind:

four    two "e"         
four    two "e"         
three   three   "e"         
one three   "e"         
zero    six one zero    "e" 
two two "e"         
two two "e"         
two two "e"         
two two "e"         
two two "e"         
four    zero    "e"         
two two "e"         
three   one three   "e"     
three   "e"             
four    one "e"         
three   one "e"         
two one one zero    two "e"
two five    "e"         
three   four    two "e"     
zero    five    "e"         
three   "e"             
three   three   "e"         
three   "e"             
one one "e"         
three   two "e"         
one one "e"         
three   two zero    zero    zero    "e"
three   three   "e"         
three   one "e"         
three   two "e"         

I need the output to be something like: {"four":[{2:"two", 3:"one",2:"exit"},{...}],"three":[{...}]}

The numbers above are basically how many times does the transition to a particular state happens..

I am using python for this.

Answer to the usual question "what have you tried?": "I tried a couple for approaches but they did not quite deliver, so I hope one of the answers will help clarifying things a bit".

Thanks a lot.

EDIT, updated to show the full matrix.

Answer 1

You haven't given a matrix of transitions (these would be probabilities), but rather a sequence of observed transitions that result from an underlying Markov model.

Unless you have an infinite number of these observations, then you can't reconstruct the underlying transition parameters exactly. You can however, choose the transitions such that your sequence of observables is the most likely. If I understand your question, you should look into solutions of Hidden Markov Models. A freely available python module GHMM can be found here.

Answer 2

Here's an idea: Instead of trying to create {"four":[{2:"two", 3:"one",2:"exit"},{...}],"three":[{...}]} (which isn't quite legal in python), try to create {"four":[{"two":2, "one":3, "exit":2},{...}],"three":[{...}]} (notice the change in the order in the inner dictionary).

Iterate over the matrix, for each line:
  if the first word isn't in the big dictionary, add it.
  if the second word isn't in its sub-dictionary, add it, otherwise add 1 to its value.