Why is it not possible to get a Py_buffer from an array object?

Question

The python documentation on array clearly states that the array conforms to the buffer interface. It even suggest not using the buffer_info() method. But when I try to get a Py_Buffer from C/C++ code with PyObject_GetBuffer() or use python's memoryview, I get a failure.

For example, in python (I use version 2.7):

>>> a = array.array('c')
>>> memoryview(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot make memory view because object does not have the buffer interface

In fact, when I search python's code base, only bytearrayobject (bytearray), memoryobject (memoryview), and stringobject (str) have the required Py_TPFLAGS_HAVE_NEWBUFFER flag set on them. To my understanding, the documentation is wrong; array does not support the buffer interface.

I could use bytearray which supports the buffer interface, the problem is that I need the array's practical fromfile() method to read in a buffer that I can use in my C/C++ code.

Is there an alternative that would allow me to read a file into a buffer and use this buffer from C code, and not involve memory copies ? (I want to treat big binary files and copying is a less desirable option).

Answer 1

memoryview works only on objects that support the Python 3 buffer interface. array.array in Python 3 does, but it doesn't in Python 2.7. You might want to file a bug report for that. Simply use use bytearray (or str if you're using it read-only). Both support memoryview just fine.

Answer 2

Python 2.6+ has two different buffer interfaces, just like it has two different class types: the classic version, and the Python 3 version.

From the Python/C API Reference Manual:

Two examples of objects that support the buffer interface are strings and arrays. The string object exposes the character contents in the buffer interface’s byte-oriented form. An array can only expose its contents via the old-style buffer interface. This limitation does not apply to Python 3, where memoryview objects can be constructed from arrays, too.

In Python 2.7 code, you can work with the old-style buffers using the buffer function, and new-style buffers using memoryview. Python 3 only supports the latter.

A similar distinction exists in the Python 2 C API; PyObject_GetBuffer is for the new buffer interface, PyBuffer_FromObject/PyBuffer_FromReadWriteObject is for the old buffer interface (and should work for arrays). See the above link for more information.