positives = [b for b in A if b >= 0]
doesn't do what you think it does. b
would be a list of 7 elements, how can a list of 7 elements be greater than 0?
numpy makes this easy:
import numpy as np
import numpy.ma as ma
A = [[-9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0],
[-9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0],
[0.040896, 0.018690, 0.005620, -9999.0, 0.038722, 0.018323, -9999.0],
[0.039443, 0.017517, 0.003460, -9999.0, 0.035526, 0.011692, -9999.0],
[-9999.0, 0.017963, 0.005264, -9999.0, 0.03788, 0.014316, -9999.0]]
A = np.array(A)
sigmas = []
for b in A:
bmask=ma.masked_array(b,mask=np.greater_equal(b,0))
b=b[bmask.mask]
print b
sigmas.append(np.std(b))
gives
[]
[]
[ 0.040896 0.01869 0.00562 0.038722 0.018323]
[ 0.039443 0.017517 0.00346 0.035526 0.011692]
[ 0.017963 0.005264 0.03788 0.014316]
>>> sigmas
[0.0, 0.0, 0.013412289355661845, 0.013828802328473713, 0.011917047544903896]
edit: in response to comment
>>> A=[[1,2,3,4,5,6,7],[2,-3,4,-3,2,1,-9]]
>>> [b for b in A if b>=0]
[[1, 2, 3, 4, 5, 6, 7], [2, -3, 4, -3, 2, 1, -9]]
Python doesn't give you an error, but it is not comparing the elements in b
to 0
, it is only comparing b
, which is evaluated as a boolean.
Here you can see explicitly what is happening:
>>> bool(b)
True
>>> True >= 0
True
For every list of 7 numbers b
in A you are just doing True >= 0
, which is always True
.
edit2: I'm an idiot and see now you were trying to use map and the problem I was talking about would be avoided. Just change G = map(sigma, zip(*A))
to G = map(sigma, A)
edit3:: you were returning i
instead of diff
. here is the code:
def sigma(A):
positives = [b for b in A if b >= 0]
if positives:
mean = sum(positives) / len(positives)
diff = ((sum([abs(i - mean)**2 for i in positives]))/(len(positives)))**(0.5)
for i in positives:
if (abs(i - mean)) > (diff*3):
return -9999.00
return diff
else:
return -9999.00
A = [[-9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0],
[-9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0],
[0.040896, 0.018690, 0.005620, -9999.0, 0.038722, 0.018323, -9999.0],
[0.039443, 0.017517, 0.003460, -9999.0, 0.035526, 0.011692, -9999.0],
[-9999.0, 0.017963, 0.005264, -9999.0, 0.03788, 0.014316, -9999.0]]
G = map(sigma, A)
which gives:
>>> G
[-9999.0, -9999.0, 0.013412289355661845, 0.013828802328473713, 0.011917047544903896]
edit4: clarified problem
def sigma(A):
positives = [b for b in A if b >= 0]
sq_err=[]
if positives:
mean = sum(positives) / len(positives)
diff = ((sum([abs(i - mean)**2 for i in positives]))/(len(positives)))**(0.5)
for i in positives:
if (abs(i - mean)) > (diff*3):
sq_err.append(-9999.00)
else:
sq_err.append(i)
else:
return [-9999.00]
return sq_err
A = [[-9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0],
[-9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0],
[0.040896, 0.018690, 0.005620, -9999.0, 0.038722, 0.018323, -9999.0],
[0.039443, 0.017517, 0.003460, -9999.0, 0.035526, 0.011692, -9999.0],
[-9999.0, 0.017963, 0.005264, -9999.0, 0.03788, 0.014316, -9999.0]]
G = map(sigma, A)
gives
>>> G
[[-9999.0], [-9999.0], [0.040896, 0.01869, 0.00562, 0.038722, 0.018323], [0.039443, 0.017517, 0.00346, 0.035526, 0.011692], [0.017963, 0.005264, 0.03788, 0.014316]]