Fast iteration over columns of a matrix with ctypes

Question

let A be a matrix

import numpy as np
A = np.matrix([[3.0, 1.0, 2.0, 0.0], [2.0, 1.0, 3.0, 1.0], [0.0, 2.0, 0.0, 3.0]])

[[ 3.  1.  2.  0.]
 [ 2.  1.  3.  1.]
 [ 0.  2.  0.  3.]]

I am facing a complex library (based on the ctypes interface) that is excepting me giving pointers to the columns of the matrix, e.g.:

import ctypes

for j in range(0,4):
    a = np.copy(A[:,j])
    lib.DoSomething(a.ctypes.data_as(ctypes.POINTER(ctypes.c_double)))

Obviously I am keen to avoid the copying of the column into the variable a. I guess there are numerous clever ideas? Maybe I should transpose and copy the matrix? Or is there a way I can enfore to store it columnbased?

Thomas

Answer 1

You can store the matrix in Fortran order, so the columns are contiguous. Then just pass a view to that contiguous column.

A = np.array([[3.0, 1.0, 2.0, 0.0], [2.0, 1.0, 3.0, 1.0], [0.0, 2.0, 0.0, 3.0]], order='F')
for j in range(0, 4):
    a = A[:, j]

Also, avoid using np.matrix unless you really really need it. It is a bug magnet.

Answer 2

So numpy will only store your data in row-major order. If you want to avoid the copy operation you can do:

EDIT: So the above statement is nonsense and the simple answer it just to set order='F' as in the above answer.

import numpy as np

# avoid matrix and define the dtype at the start so you don't have an extra copy
A = np.array([[3.0, 1.0, 2.0, 0.0], [2.0, 1.0, 3.0, 1.0], [0.0, 2.0, 0.0, 3.0]],dtype="float64",order="F")

for row in A.transpose():
    lib.DoSomething(np.ctypeslib.as_ctypes(row))