awk two files based on 1st & 2nd column

Question

I'm trying to merge a new file with an old one.

There is a unique key on the primary column, then a separator '=' and a value.

• In case primary key exist in both file I must keep the old value and if the new one is different add a comment near the line.
• In case primary key exist only in old, keep it.
• In case primary key exist only in new, insert it

For example:

In the old file:

$ cat oldfile.txt
VAR NAME ONE = FOO
TWO BAR = VALUE 
; this is a comment

In the new one:

$ cat newfile.txt
TWO BAR = VALUE
; this is a comment
VAR NAME ONE = BAR
NEW = DATA

Desired output :

$ cat output.txt
VAR NAME ONE = FOO
;new value:
; VAR NAME ONE = BAR
TWO BAR = VALUE 
; this is a comment
NEW = DATA

I've tried to deal with diff but it work only line by line, I'm pretty sure awk can do it.. but I'm not an expert with awk. I can write something in ksh to do the job, but I'm pretty sure there is an other way quicker and simpler.

Please note, order of line in previous and new file can change and I'm on AIX (Unix), not Linux.

Thanks for your help :)

EDIT:

I did not precise in the first post, new comments must be kept if they are not already present in the previous file.

Answer 1

Using awk:

awk '
BEGIN {FS=OFS="="}
NR==FNR {
     line[$1] = $2;
     next
}
($1 in line) && ($2!=line[$1]) {
    print $0; 
    print ";new value:"; 
    print "; "$1, line[$1];
    delete line[$1]; 
    next
}
($1 in line) && ($2==line[$1]) {
    print $0;
    delete line[$1]; 
    next
}1
END {
    for (k in line) 
        print k, line[k]
}' newfile oldfile

Output:

VAR NAME ONE = FOO
;new value:
; VAR NAME ONE = BAR
TWO BAR = VALUE
; this is a comment
NEW = DATA

Answer 2

Perl solution. First, it reads the new file into a hash. Then it goes over the old one and consults the hash for the changes. You did not specify what to do with comments in the new file, you have to tweak the code at the corresponding comment.

#!/usr/bin/perl
use warnings;
use strict;

my ($oldfile, $newfile) = @ARGV;

my %new;
my $last;
open my $NEW, '<', $newfile or die $!;
while (<$NEW>) {
    chomp;
    if (/^;/) {
        $new{$last}{comment} = $_; # What should we do with comments?
    } elsif (! /=/) {
        warn "Invalid new line $.\n";
    } else {
        my ($key, $value) = split /\s* = \s*/x, $_, 2;
        $new{$key}{value} = $value;
        $last = $key;
    }
}

open my $OLD, '<', $oldfile or die $!;
while (<$OLD>) {
    chomp;
    if (/^;/) {
        print "$_\n";
    } elsif (my ($key, $value) = split /\s* = \s*/x, $_, 2) {
        if (exists $new{$key}) {
            print "$key = $value\n";
            if ($new{$key}{value} ne $value) {
                print ";new value:\n";
                print "; $key = $new{$key}{value}\n";
            }
        } else {
            print "$key = $value\n";
        }
        delete $new{$key};
    } else {
        warn "Invalid old line $.\n";
    }
}
for my $key (keys %new) {
    print "$key = $new{$key}{value}\n";
}