Store empty numpy array in h5py

Question

I'd like to write some data to a HDF5 file (since I have huge datasets and was told that HDF5 works well with those kinds of things).

I have a Python 2.7 dictionary with some values and some numpy arrays. What I'd like to do is simply dump that dictionary into the HDF5. No groups or whatever, just put the key-value pairs into the HDF5.

However, using h5py, if I write an empty array (or list) into the file, I get:

>>> file["test"] = np.array([])
ValueError: zero sized dimension for non-unlimited dimension (Invalid arguments to routine: Bad value)

I can't believe that HDF5 wouldn't allow me to put empty arrays into it. It just so happens that sometimes my list is empty. Can't help it.

What am I missing?

Thanks :-)

Answer 1

As mentioned in the comments, arrays with size-zero dimensions are not supported in h5py 2.0.0. As far as I can tell, this feature seems to have been added in this pull request.

Answer 2

This is what I use (it might be unnecessary for you since you don't require groups but you might change you mind):

def save_group(outfile, d, group):
    with h5py.File(outfile, 'a') as g:
        for key in d:
            value = d[key]
            try:
                group.require_dataset(key, value.shape, value.dtype)
            except TypeError as e:
                del group[key]
                group.create_dataset(key, value.shape, value.dtype)
            group[key][...] = value
    return

To get rid of the group parameter:

group = g.require_group('/')