Using MFDataset to combine netcdf files in python

Question

I am trying to combine netcdf files, but it contifuously shows " File "CBL_plot.py", line 11, in f = MFDataset(fili) File "utils.pyx", line 274, in netCDF4.MFDataset.init (netCDF4.c:3822) IOError: master dataset THref_11:00.nc does not have a aggregation dimension."

So, I checked only one netcdf files and the information of a netcdf file is as below:

float64 th_ref(u't',) unlimited dimensions = () current size = (30,)

It looks there is no aggregation dimension. However, I would like to combine those netcdf files rather than just using one by one. Is there any way to create aggregation dimension to make this MFData set work?

Below is the python code I used:

    import numpy as np
    from netCDF4 import MFDataset
    varn = 'th_ref'
    fili = THref_*nc'
    f    = MFDataset(fili)
    Th  = f.variables[varn]
    Th_ref=np.array(Th[:])
    print Th.shape

I will really appreciate any help, idea, and hint.

Thank you, Isaac

Answer 1

Short answer: MFDataset can only aggregate along the slowest varying dimension in your files.

Longer answer: In the netcdf4-python documentation of MFDataset it says "Open a Dataset spanning multiple files, making it look as if it was a single file. Variables in the list of files that share the same dimension (specified with the keyword aggdim) are aggregated. If aggdim is not specified, the unlimited is aggregated. Currently, aggdim must be the leftmost (slowest varying) dimension of each of the variables to be aggregated."

So MFDataset works by aggregating along the slowest varying dimension in the existing files. So if you have a bunch of files that are snapshots of the same logical dataset at different times, and you want to aggregate in time, you need to have a time dimension in each of the files. If the time of the data is simply encoded in the file name, there is currently no way to use MFDataset to aggregate.