Compare two txt files containing Hex numbers

Question

I have two files A.txt and B.txt containing Hex values. A.txt looks like:

blah blah blah ;AA=0012FF34, BB=0012FC0
blah blah blah ;AA=00120F54
blah blah blah ;CC=00978E4A
blah blah blah ;AA=007649A4, BB=0032FFF, CC=00F5FC6

and B.txt is something like:

b-base    b-size
00020000 00001000 blah blah blah
00030000 00001000 blah blah blah
00040000 00001000 blah blah blah
000E0000 00001000 blah blah blah
000F0000 00005000 blah blah blah

How can I print the lines of A.txt in which the value of AA, BB, CC, or DD in that line is inside one of the following boundaries:

00020000< <00020000+00001000
00030000< <00030000+00001000
00040000< <00040000+00001000
000E0000< <000E0000+00001000
000F0000< <000F0000+00005000

Point: The first part of A.txt (before ";") can have any arbitrary length.

Answer 1

I wrote some Perl that should do the job, as it seemed like a bit much to do in AWK or sed. It constructs a hash of the ranges in B.txt and prints out any lines from A.txt, where any one of the values in the comma separated list lies within any of the ranges (currently >= start and < end). Note that this was a bit of a learning exercise for me, so I welcome any constructive feedback.

#!/usr/bin/env perl

use strict;
use warnings;

open my $fh,"<","B.txt" or die "couldn't open file: $!";
<$fh>; # skip first line
my @range;
while (<$fh>) {
    my @F = split;
    push @range, [hex($F[0]), hex($F[0]) + hex($F[1])];
}
close $fh;

open $fh,"<","A.txt" or die "couldn't open file: $!";
while (<$fh>) {
    my $match = 0;
  OUTER:
    for (split ',', (split ';')[1]) {
        chomp (my $val = (split '=')[1]);
        $val = hex $val;
        for my $ref (@range) {
            if ($val >= $$ref[0] && $val < $$ref[1]) {
                $match = 1;
                last OUTER;
            }
        }
    }
    print if $match;
}

Answer 2

Stunned no-one else did it in awk!!! What's going on?

gawk '
   BEGIN {j=0}
   FNR==NR {
      lo[j] = strtonum("0x"$1)
      hi[j] = strtonum("0x"$2)+lo[j]
      j++
      next
   }
   {
      line=$0                 # Save line in case we need to print it
      sub(/.*;/,"",$0)        # Remove everything before semicolon
      split($0,a,",")         # Split rest on commas into array a[]
      for(x in a){            # Iterate through all AA=, BB=, CC=
        sub(/.*=/,"",a[x])    # Remove everything up to and including = sign
        d=strtonum("0x"a[x])  # Convert to decimal
        for(y in lo){
           if((d>=lo[y])&&(d<=hi[y])){print line;break}
        }
      }
   }
   ' B.txt A.txt

Answer 3

I also felt this was beyond the scope of awk and sed, so I did the same as Thomas, but in python. Feel free to use this or improve on it any way you see fit.

def main(a, b):
    with open(b) as bf:
        data = bf.readlines()[1:]
        limits = get_limits(data)
    with open(a) as af:
        for line in af:
            maybe_print(line, limits)

def get_limits(data):
    limits = []
    for line in data:
        base, size = line.split(' ')[0:2]
        limits.append((int(base, 16), int(size, 16)))
    return limits

def maybe_print(line, limits):
    data = line.split(';')[1]
    data = data.split(', ')
    for datum in data:
        value = int(datum.split('=')[1], 16)
        for base, size in limits:
            if value > base and value < base + size:
                print line,
                return

if __name__ == '__main__':
    import sys
    main(sys.argv[1], sys.argv[2])

I run it like this: python <scriptname> A.txt B.txt

Answer 4

My Ruby implementation, for the record...

ranges = File.readlines("B.txt").grep(/^([\dA-F]+)\s+([\dA-F]+)/i){ ($1.hex)..($1.hex+$2.hex) }
File.open("A.txt") do |f|
    f.each_line do |line|
        numbers = line.scan(/\b\w+=([\dA-F]+)/).collect{|x| x.first.hex }
        puts line if numbers.detect{|x| ranges.detect{|y| y === x } }}
    end
end