Extract elements of an N-dimensional array within a list in Python

Question

I have the following problem. I save a large amount of data within a class. Most of these data are time dependent and in the most complex cases, the variables are 3-dimensional array.

Because list are quite flexible (no need of explicit declaration), I wanted to use them to encapsulate my N-dimensional arrays and thus, use lists to keep the time dependence information.

Here a typical example of what I have for the element t=0, t=2 and t=3 of my list which is within the history class (a simple matrix of float64):

history.params[0]
array([[ 1.        ,  2.        ,  1.        ],
       [ 1.        ,  2.        ,  1.        ],
       [ 0.04877093,  0.53176887,  0.26210472],
       [ 2.76110434,  1.3569246 ,  3.118208  ]])

history.params[2]
array([[ 1.00000825,  1.99998362,  1.00012835],
       [ 0.62113657,  0.47057772,  5.23074169],
       [ 0.04877193,  0.53076887,  0.26210472],
       [ 0.02762192,  4.99387138,  2.6654337 ]])
history.params[3]
array([[ 1.00005473,  1.99923187,  1.00008009],
       [ 0.62713657,  0.47157772,  5.23074169],
       [ 0.04677193,  0.53476887,  0.25210472],
       [ 0.02462192,  4.89387138,  2.6654337 ]])

Now, How do I do to read/extract, all elements at given coordinate (x,y) of my matrix, for all the time indexes t?

I tried by doing:

history.params[:][0][0]

and I get

array([ 1.,  2.,  1.])

Actually whatever the place of the colon, I always get the same values, which correspond to the first row of my matrix:

"history.params[0][:][0]" returns "array([ 1.,  2.,  1.])" in the shell

"history.params[0][0][:]" returns "array([ 1.,  2.,  1.])"

Why Python is not able here to distinguish the elements of the matrix, from the elements of the list? What is the best solution?

Of course, I can write some loops and create a new variable that reorganize my data, but it is a bit a waste of energy. I am certain that it exists an elegant solution.

PS: I am going to 'Cythonize' my code at some point, so if you have an optimized solution for Cython to store these variables, I am very happy to hear it as well.

Answer 1

I would suggest using a numpy.array array rather than nested lists.

import numpy as np

# Create some data which would be equal to your "params"
a = np.array([[[1, 2, 3],
               [4, 5, 6],
               [7, 8, 9]],
              [[11, 12, 13],
               [14, 15, 16],
               [17, 18, 19]]])

print(a[0])
# [[1 2 3]
#  [4 5 6]
#  [7 8 9]]

print(a[:, 0, 0])
# [1, 11]

print(a[:, 0:2, 0])
# [[1, 4] 
#  [11, 14]]

Furthermore numpy can be combined with Cython as given here.

Answer 2

Why Python is not able here to distinguish the elements of the matrix, from the elements of the list? What is the best solution?

Because history.params[0] is a list of lists, so history.params[0][0] is a list, so history.params[0][0][:] is list[:] which is a copy of the inner list. Similarly, history.params[0][:] is a copy of the list of lists, so history.params[0][:][0] is (copy of the list of lists)[0] which is again the first row of your inner list, but in the copy of the 2D array.

If you want to flatten your list, ie store a 2D array as one list, each element at (n,m) of 2D array of size NxM becomes element (n*M + m) in your flattened array. So in a 4x3 as you have posted, element (0,0) is element 0 of flat list, element (2,1) is element 2*3+1 = 7, and so forth.

You can extend that to 3D arrays: array of size KxNxM, you would have index (k,n,m) is element (k*N*M + n*N + m); and similarly for higher dimensionalities.