How does Labview save cluster data in a binary file and how do I read it out in MATLAB?

Question

I have a very large number of files that were saved in binary in Labview, where each column is a timestamp cluster followed by a vector of singles.

I read each data file into Matlab r2013a using

fid = fopen(filename);
data = fread(fid,[N M],'*single',0,'b');
fclose(fid);

where I pre-calculate the size of the input array N,M. Since I know what the data is supposed to look like, I have figured out that data(1:5,:) is where the timestamp data is hidden, but it looks like something like this for M = 1:

[0 -842938.0625 -1.19209289550781e-07 0 4.48415508583941e-42]

The first element is always 0, the second element decreases monotonically with a constant step size, the third seems to be bistable, flipping back and forth between two very small values, the fourth is always 0, and the fifth is also constant.

I'm assuming it has something to do with how Labview encodes dates, but my google-fu has not helped me figure that out.

To make this a more general question, then:

How does Labview encode a timestamp cluster when it saves to a binary file, and how can I read it out and translate it into a meaningful number in another programming language, such as Matlab?

EDIT: For posterity, here is my final code (appended to the code above):

datedata = data(5:-1:1,:);
data(1:5,:) = [];

dms   = typecast(reshape(datedata(2:3,:),[],1),'uint64');
dsecs = typecast(reshape(datedata(4:5,:),[],1), 'int64');

timestamp = datenum(1904,1,1) + (double(dsecs) + double(dms)*2^-64)/(3600*24);

In the code @Floris posted from Mathworks, they typecast straight to double, but when I tried that, I got garbage. In order to get the correct date, I had to first convert to integer and then to double. Since my bottleneck is in the fread line (0.3 seconds to read off of an external disk), the extra typecast step is miniscule in the grand scheme of things.

The extra column, 4.5e-42, converts to an integer value of 3200, the number of values in the subsequent vector of singles.

Answer 1

This is not a complete answer, but it should help (I don't have either Labview or Matlab available at home so I can't check this right now).

There is an article at http://www.mathworks.com/matlabcentral/newsreader/view_thread/292060 that describes a similar question. Couple of useful bits of information I extracted from that:

1. Time stamp is a double (not single)
2. Need to flip the order of bytes (little vs big endian) to make sense of things

There is a useful comment:

Note that the LabView time convention is miliseconds since Jan 1 1904. Here is one approach (may contain errors but will point you in the right direction),

The following code snippet is also given:

%% Read in date information
[ fid, msg ] = fopen(FileName, 'r') ;
NColumns = 60 ; % Number of data columns - probably different for your
dataset!
[a, count] = fread(fid, [ NColumns Inf], '*single') ; % Force data to
be read into Matlab workspace as singles
a = a' ; % Convert to data in columns not rows
% The last two columns of a are the timestamp
b = fliplr(a(:, end-1:end)) ; % Must swap the order of the columns
d = typecast(reshape(b',[],1), 'double') ; % Now we can can convert to
double
time_local = datenum(1904, 1, 1) + d/(24*3600) ; % Convert from
seconds to matlab time format
fclose(fid) ;

It looks believable to me. Let me know if it works - if not, I may be able to help debug in the morning...

Answer 2

A LabVIEW timestamp is a 128-bit type consisting of a signed 64-bit integer measuring the offset in seconds since the LabVIEW epoch (January 1, 1904 00:00:00.00 UTC), and an unsigned 64-bit integer measuring the fractional second. Source: ni.com.

The byte order of the file however may be platform dependent. For example the time 8:02:58.147 AM July 3 2013 EDT may be stored as:

0x 00000000CDF9C372 25AA100000000000 (big/network)

or as

0x 000000000010AA25 72C3F9CD00000000 (little)