Using dictionary for matrix factorization and "nan" came up in dictionary.values()

Question

I was learning and trying to build a toy recommend system using LFM(latent factor model). So I find something about matrix factorization in this page (http://www.quuxlabs.com/blog/2010/09/matrix-factorization-a-simple-tutorial-and-implementation-in-python/)

The code inside that page can run perfectly. But in my work the matrix should be a sparse one since lots of element remain blank after initializing. So I rewrite it using dictionary and everything screws up.

Here is the codes given in the web page:

import numpy

def matrix_factorization(R, P, Q, K, steps=5000, alpha=0.0002, beta=0.02):
    Q = Q.T

    for step in xrange(steps):
        for i in xrange(len(R)):
            for j in xrange(len(R[i])):
                if R[i][j] > 0:
                    eij = R[i][j] - numpy.dot(P[i,:],Q[:,j])
                    for k in xrange(K):
                        P_temp = P[i][k]
                        Q_temp = Q[k][j]

                        P[i][k] = P_temp + alpha * (2 * eij * Q_temp - beta * P_temp)
                        Q[k][j] = Q_temp + alpha * (2 * eij * P_temp - beta * Q_temp)

        e = 0
        for i in xrange(len(R)):
            for j in xrange(len(R[i])):
                if R[i][j] > 0:
                    e = e + pow(R[i][j] - numpy.dot(P[i,:],Q[:,j]), 2)
                    for k in xrange(K):
                        e = e + (beta/2) * (pow(P[i][k],2) + pow(Q[k][j],2))
        if e < 0.001:
            break
    return P, Q.T

if __name__ == '__main__':
    R = [
         [5,3,0,1],
         [4,0,0,1],
         [1,1,0,5],
         [1,0,0,4],
         [0,1,5,4],
        ]

    R = numpy.array(R)

    N = len(R)
    M = len(R[0])
    K = 2

    P = numpy.random.rand(N,K)
    Q = numpy.random.rand(M,K)

    nP, nQ = matrix_factorization(R, P, Q, K)
    nR = numpy.dot(nP, nQ.T)

This code works all right. So I write the following code:

import random

def matrix_factorization(R, P, Q, K,steps=5000, alpha=0.0002, beta=0.02):
    for step in xrange(steps):
        print 'step',step
        step += 1
        for i in R.keys():
            for j in R[i].keys():
                eij = R[i][j] - sum([x * y for x in P[i] for y in Q[j]])
                for k in xrange(K):
                    P_temp = P[i][k]
                    Q_temp = Q[j][k]

                    P[i][k] = P_temp + alpha * (2 * eij * Q_temp - beta * P_temp)
                    Q[k][j] = Q_temp + alpha * (2 * eij * P_temp - beta * Q_temp)


        e = 0
        for i in R.keys():
            for j in R[i].keys():
                e += pow(R[i][j] - sum([x * y for x in P[i] for y in Q[j]]), 2)
                for k in xrange(K):
                    e += (beta/2) * (pow(P[i][k],2) + pow(Q[k][j],2))

        if e < 0.001:
            break
    return P,Q


if __name__ == '__main__':    
    R = {0:{0:5,1:3,3:1},
         1:{0:4,3:1},
         2:{0:1,1:1,3:5},
         3:{0:1,3:4},
         4:{1:1,2:5,3:4}
         }

    N = len(R.keys())
    M = 4
    K = 4

    P = dict()
    Q = dict()

    for i in xrange(N):
        P[i] = [random.random() for x in xrange(K)]

    for j in xrange(M):
        Q[j] = [random.random() for x in xrange(K)]

    P,Q = matrix_factorization(R,P,Q,K)
    Rij = dict()

These two sections should have same function and the structures are just the same. BUT!What my code returns is:

OverflowError: (34, 'Result too large')

or after calculating P and Q shows:

P
Out[5]: 
{0: [nan, nan, nan, nan],
 1: [nan, nan, nan, nan],
 2: [nan, nan, nan, nan],
 3: [nan, nan, nan, nan],
 4: [nan, nan, nan, nan]}

Q
Out[6]: 
{0: [nan, nan, nan, nan],
 1: [nan, nan, nan, nan],
 2: [nan, nan, nan, nan],
 3: [nan, nan, nan, nan]}

I just can't figure it out why and the very sad fact is I have finish my recommend system using this method. Could you help me find the reason why this happened? Thank you very much for your time!

Answer 1

I changed the following line in the function matrix_factorization Q[k][j] = Q_temp + alpha * (2 * eij * P_temp - beta * Q_temp) to Q[j][k] = Q_temp + alpha * (2 * eij * P_temp - beta * Q_temp) Then the modified code seems to work well.

I modified the function matrix_factorization as follows, then the result seems right.

def matrix_factorization(R, P, Q, K, steps=5000, alpha=0.0002, beta=0.02):
    for step in xrange(steps):
        for i in R.keys():
            for j in R[i].keys():
                eij = R[i][j] - sum([P[i][k] * Q[j][k] for k in xrange(K)])
                for k in xrange(K):
                    P_temp = P[i][k]
                    Q_temp = Q[j][k]

                    P[i][k] = P_temp + alpha * (2 * eij * Q_temp - beta * P_temp)
                    Q[j][k] = Q_temp + alpha * (2 * eij * P_temp - beta * Q_temp)

        e = 0
        for i in R.keys():
            for j in R[i].keys():
                e += pow(R[i][j] - sum([P[i][k] * Q[j][k] for k in xrange(K)]), 2)
                for k in xrange(K):
                    e += (beta/2) * (pow(P[i][k], 2) + pow(Q[j][k], 2))
        if e < 0.001:
            break
    return P,Q