MATLAB Save Cell Arrays to NetCDF File

Question

I have a .csv file that I've imported into MATLAB using the import data function as column vectors. Each column in the .csv file is now a cell array of size 7286x1. How can I save each of these column vectors as a variable into a netcdf file?

ncid = netcdf.create('WI_O3.nc', 'NOCLOBBER');
dimid = netcdf.defDim(ncid, 'my_dim', 7286);
SiteID = netcdf.defVar(ncid, 'AQS_SITE_ID', 'double', dimid);
netcdf.endDef(ncid);
netcdf.putVar(ncid, SiteID, AQS_SITE_ID);
    % Error using netcdf.putVar (line 56)
    % Expected DATA to be one of these types:
    % 
    % double, single, uint8, uint16, uint32,
    % uint64, int8, int16, int32, int64, char
    % 
    % Instead its type was cell.

How can I convert all my cells to a string or double or whatever that would make it work? Some of my cells are composed of words (strings) while others are numbers (double)

Is there a faster way to just save all of the variables in the workspace into one .nc file (keeping all the variable names the same is fine)?

Answer 1

It's not exactly what you asked, but here's a script that writes all the {double,single,char} variables in a structure to a netcdf file. It's not the most general code, because I'm only handling up to 3D variables (although that would be easy to extend). I'm posting it here because I think it might be of general use and interest, and perhaps someone will even post a better version.

I also compress all the variables that are not scalar or char.

function struct2nc(x,ncfile,ncfiletype,deflate_lev)
% STRUCT2NC writes all float,double and character vars to netcdf
% Usage: struct2nc(x,ncfile,[ncfiletype],[deflate_lev])
% x = structure
% ncfile = name of netcdf output file (e.g. 'test.nc')
% ncfiletype = netcdf file type (e.g. 'classic','netcdf4_classic')
% deflate_lev = deflate level (0-9, 0 is none)
%
% This function writes all 'double','single' and 'char' variables
% to NetCDF using the native Matlab NetCDF interface.  It skips all
% other classes in the struct (e.g. structs, cell arrays, etc).  It
% also only handles scalar, 1D, 2D, and 3D arrays currently, although
% this could easily be extended.

if nargin==2,
    ncfiletype='classic';
    deflate_lev=0;
elseif nargin==3;
    switch ncfiletype
        case {'netcdf4','netcdf4_classic'}
            deflate_lev=6;
        otherwise
            deflate_lev=0;
    end
end
s = fieldnames(x);
k=0;
% create variables first, but don't write data
for i=1:length(s)
    vname=char(s(i));
    var=x.(vname);
    vtype = class(var);
    vshape = size(var);
    ndims = length(vshape);
    vlen = length(var(:));
    switch vtype;
        case {'double','single'},
            if vlen==1,
                nccreate(ncfile,vname,...
                    'Datatype',vtype,'format',ncfiletype);
                k=k+1;
                vnames{k}=vname;
            else
                if min(vshape)==1,
                    nccreate(ncfile,vname,...
                        'Datatype',vtype,...
                        'DeflateLevel',deflate_lev,...
                        'Dimensions',{[vname '1'] vlen},...
                        'format',ncfiletype);
                    k=k+1;
                    vnames{k}=vname;
                elseif ndims==2,
                    nccreate(ncfile,vname,...
                        'Datatype',vtype,...
                        'DeflateLevel',deflate_lev,...
                        'Dimensions',{[vname '1'] vshape(1) [vname '2'] vshape(2)},...
                        'format',ncfiletype);
                    k=k+1;
                    vnames{k}=vname;
                elseif ndims==3,
                    nccreate(ncfile,vname,...
                        'Datatype',vtype,...
                        'DeflateLevel',deflate_lev,...
                        'Dimensions',...
                        {[vname '1'] vshape(1) [vname '2'] vshape(2) [vname '3'] vshape(3)},...
                        'format',ncfiletype);
                    k=k+1;
                    vnames{k}=vname;
                else,
                    disp('Skipping variable with more than 3 dimensions');
                end
            end
        case {'char'},
            nccreate(ncfile,vname,...
                'Datatype',vtype,...
                'Dimensions',{[vname '1'] vlen},.....
                'format',ncfiletype);
            k=k+1;
            vnames{k}=vname;
        otherwise,
            disp(['skipping ' vname])
    end
end
%write all the data at the end
for i=1:length(vnames)
    ncwrite(ncfile,vnames{i},x.(vnames{i}));
end

Answer 2

The netcdf.* functions are lower level functions for direct access to the NetCDF library. Use ncwrite along with nccreate.

Have a look at the example for NCCREATE and example for NCWRITE which shows you how to create the variable, along with the required dimensions and then write to it. (The second example in the NCWRITE doc page should be a good starting point).

The NetCDF format does not support 'cells'. Create an ND variable in MATLAB as a normal numeric array and then write it into the NetCDF file.

Answer 3

Interesting problem. AFAIK, NetCDF3 and NetCDF4 formats do not have anything akin to Matlab's cell array, which can hold different types of data in each cell. Neither, unfortunately, does Matlab currently (1/18) support the ability to read and write NetCDF4's variable-length strings (even though it has recently added a string datatype).

So, two possible solutions.

1. convert all numeric data to strings with sprintf(...). These will unfortunately have to be fixed length strings, which would need to be long enough to retain your necessary precision. Then store the data as a fixed-length 2-D char array.

2. Encode your string data with numeric values. For example, if you have strings of "Ozone", "CO2", "Methane", replace these with, say, -999, -998, and -997 respectively. (Obviously, choose numeric values that cannot occur in your actual data.) This is similar to how you need to encode nan's, flagged by the attribute _FillValue. You should also store the flags and corresponding original strings, either as attributes or as a separate array of chars and associated flag values.