How do I read a text file with variable hexadecimal value columns in matlab?

Question

I have a rather large text file (over 16,000 lines) that has the following format:

#ID     #Line Num       #Var Col Length Values (HEX):
45      00001           FFFF FFFF 0000 0000
45      00002           0000 0000 FFFF FFFF
47      00003           AAAA 1111 AAAA 1111 AAAA 1111
49      00004           BBBB 2222 

Note: This is obviously made up data, as there are more HEX values in the actual file.

In Matlab I tried using the textscan command of a single line:

fp = fopen(filePath, 'rt');
readLine = fgetl(fp);
[ignored, pos] = textscan(readLine, '%d');
values = textscan(readLine(pos+1:end), '%x');

I get an error of a badly formatted string. I'm assuming that textscan doesn't support conversion of hex values. I've also tried the solution found here:

Problem (bug?) loading hexadecimal data into MATLAB

but that also doesn't seem to work as well. I'm trying to avoid converting each Hex Value individually (somewhat the solution I have implemented now) as this takes a very long time to do. How do I scan/parse variable column width hexadecimal values from a text file?

Answer 1

You can use the following approach read the text file into a cell array of strings and split it into separate values using regexp:

fp = fopen(filePath, 'rt');
C = textscan(fp, '%s', 'CommentStyle', '#', 'Delimiter', '');
C = regexp(C{:}, '\w+', 'match');
fclose(fp);

This should yield a cell array of cell arrays that, for your example, would look like this:

C =
    {'45'    '00001'    'FFFF'    'FFFF'    '0000'    '0000'}
    {'45'    '00002'    '0000'    '0000'    'FFFF'    'FFFF'}
    {'47'    '00003'    'AAAA'    '1111'    'AAAA'    '1111'    'AAAA'    '1111'}
    {'49'    '00004'    'BBBB'    '2222'}

You can manipulate the resulting cell array to your liking. For instance, discard the first two columns in each row, and convert everything to decimal:

result = cellfun(@(x)hex2dec(x(3:end)), C, 'UniformOutput', false)

Answer 2

You can use sscanf instead:

fp = fopen(filePath, 'rt');
readLine = fgetl(fp);
results = sscanf(readLine, '%x');

This will give you a column of converted hex values in decimal. If you need the values in a single row simply transpose it:

results = results'

This assumes that the first two terms are also Hex Values as well. If you need to separate the first two values as double and then the rest hex do the following:

fp = fopen(filePath, 'rt');
readLine = fgetl(fp);
[ignored, pos] = textscan(readLine, '%d');
results = sscanf(readLine(pos+1:end), '%x');