Can I statically type a h5file array in Cython?

Question

I develop a library that uses Cython at a low level to solve flow problems across 2D arrays. If these arrays are numpy arrays I can statically type them thus avoiding the Python interpreter overhead of random access into those arrays. To handle arrays of sizes so big they don't fit in memory, I plan to use hd5file Arrays from pytables in place of numpy, but I can't figure out if it's possible to statically type a CArray.

Is it possible to statically type hd5file CArrays in Cython to avoid Python interpreter overhead when randomly accessing those arrays?

Answer 1

If you use the h5py package, you can use numpy.asarray() on the datasets it gives you, then you have a more familiar NumPy array that you already know how to deal with.

Please note that h5py had a bug related to this until a couple years ago which caused disastrously slow performance when doing asarray() but this was solved so please don't use a very old version if you're going to try this.