Check for Nan values in the column of an ND-Array and Remove them

Question

the code below was written to check for NaN values in a Python ND-Array column. If there is a NaN in either temparr1 or temparr2, we remove the corresponding row from both of them. The problem is, it doesn't seem to work. Could you please help me out?

        temparr1=arr[index[indexkey]][:]// We get a column from arr, an nd-array of size 0 to 9470
        temparr2=arr[index[secondIndexKey]][:]// Same as above, but with the next column
        rwc=range(0,len(arr)) We get a bit vector of a sort to check.
        for i in range(0,len(arr)):
            if(isnan(temparr1[i]) or isnan(temparr2[i]) ):
                rwc = rwc[:i-1]+rwc[i+1:] // Remove the value from the bit Vector for a NaN value in the arrays.
                print i
        temparr1 = []
        temparr2 = []
        for i in rwc:
            temparr1.append(arr[index[indexkey]][i])
            temparr2.append(arr[index[secondIndexKey]][i])// Extract the data for the corresponding values in RWC and get them into the temparrs.

Can someone tell me why it is not working, why I still am getting NaNs??

An Array looks like : [99,242,122,nan,42,nan,414,................]

Answer 1

After rwc=range(0,len(arr)) you have len(rwc)=len(arr), so in the line rwc = rwc[:i-1]+rwc[i+1:] you expect that i is the same index for rwc and arr.

However after you do rwc = rwc[:i-1]+rwc[i+1:] you get a list of smaller length (len(rwc) = len(arr) -2), so during next iteration you start removing wrong elements from your list.

Also I suspect that you intended to do rwc = rwc[:i]+rwc[i+1:], which is another bug

As far as I understand you tried to do something like this:

X=arr[index[indexkey]]
Y=arr[index[secondIndexKey]]

temparr1 = []
temparr2 = []
for i in range(len(X)):  #I assume len(X)=len(Y)
    if not (isnan(X[i]) or isnan(Y[i])):
        temparr1.append(X[i])
        temparr2.append(Y[i])