group a bunch of files by day

Question

I do have a bunch of files containing atmospheric measurements in one directory. Fileformat is NetCDF. Each file has a timestamp (variable 'basetime'). I can read all files and plot individual measurement events (temperature vs. altitude). What I need to do next is "group the files by day" and plot all measurements taken at one single day together in one plot. Unfortunately I have no clue how to do that. One idea is to use the variable 'measurement_day' as it is defined in the code below. For each day I normally do have four different files containing temp. and altitude. Ideally the data of those four different files should be grouped (e.g. for plotting) I hope my question is clear. Can anyone please help me.

EDIT: I try to use a dictionary now but I have trouble to determine whether one entry already exists for one measurement day. Please see edited code below

from netCDF4 import Dataset

data ={} # was edited

for f in listdir(path):
    if isfile(join(path,f)):
        full_path = join(path,f)
        f = Dataset(full_path, 'r')
        basetime = f.variables['base_time'][:]
        altitude = f.variables['alt'][:]
        temp = f.variables['tdry'][:]
        actual_date =  strftime("%Y-%m-%d %H:%M:%S", gmtime(basetime))
        measurement_day =  strftime("%Y-%m-%d", gmtime(basetime))
        # check if dict entries for day already exist, if not create empty dict  
        # and lists inside
        if len(data[measurement_day]) == 0:
             data[measurement_day] = {}
        else: pass
        if len(data[measurement_day]['temp']) == 0:
            data[measurement_day]['temp'] = []
            data[measurement_day]['altitude'] = []
        else: pass

I get the following error message: Traceback (most recent call last):... if len(data[measurement_day]) == 0: KeyError: '2009/05/28'

Answer 1

Can anyone please help me.

I will try. Though I'm not totally clear on what you already have.

I can read all files and plot individual measurement events (temperature vs. altitude). What I need to do next is "group the files by day" and plot all measurements taken at one single day together in one plot.

From this, I am assuming that you know how to plot the information given a list of Datasets. To get that list of Datasets, try something like this.

from netCDF4 import Dataset

# a dictionary of lists that hold all the datasets from a given day
grouped_datasets = {}

for f in listdir(path):
    if isfile(join(path,f)):
        full_path = join(path,f)
        f = Dataset(full_path, 'r')
        basetime = f.variables['base_time'][:]
        altitude = f.variables['alt'][:]
        temp = f.variables['tdry'][:]
        actual_date =  strftime("%Y-%m-%d %H:%M:%S", gmtime(basetime))
        measurement_day =  strftime("%Y-%m-%d", gmtime(basetime))

        # if we haven't encountered any datasets from this day yet...
        if measurement_day not in grouped_datasets:
            # add that day to our dict
            grouped_datasets[measurement_day] = []

        # now append our dataset to the correct day (list)
        grouped_datasets[measurement_day].append(f)

Now you have a dictionary keyed on measurement_day. I'm not sure how you are graphing your data, so this is as far as I can get you. Hope it helps, good luck.