Using grep invert function to compare two md5 hash lists

Question

I’ve mounted two .dd images in Linux by using losetup and created a list of hash values for every file located on those images:

md5deep -r  -e * > winXPimage1.txt
md5deep -r  -e * > winXPimage2.txt

This is how both hash lists look like:

d41d8cd98f00b204e9800998ecf8427e  /media/74444E0F444DD510/AUTOEXEC.BAT
17d7055859d99a0d606cfaf17ae38638  /media/74444E0F444DD510/boot.ini
d41d8cd98f00b204e9800998ecf8427e  /media/74444E0F444DD510/CONFIG.SYS
88cf0ff92a4a9fa7bd9b7513b2e9e22b  /media/74444E0F444DD510/Documents and Settings/…
 Etc…

The two images contain windows XP installation files and standard programs. The second image, however, also contains a lot of pictures (jpg, png, etc.).

I want to use grep to compare the two hash lists that I’ve created and filter out all the hashes related to the .jpg files from the second image.

I've used the following command to remove all the unnecessary information from the first image:

cut -f 1 -d ' ' winXPimage1.txt > winXPimage1New.txt

So now the image1 hash list looks like this:

d41d8cd98f00b204e9800998ecf8427e
17d7055859d99a0d606cfaf17ae38638
d41d8cd98f00b204e9800998ecf8427e
etc…

I’m trying to use grep invert command to compare winXPimage1New.txt with winXPimage2.txt (contains jpeg hash values) and display all the non-matching jpg hash lines:

grep -v -f winXPimage1New.txt winXPimage2.txt/*.jpg
grep -v -f .*[.jpg] winXPimage1New.txt winXPimage2.txt

None of these commands return jpg hash values from the second image hash list. I’m just not sure where exactly do I have to put the .jpg file extension as I’m very new to Linux in general.

Answer 1

So you want to filter on .jpg files?

Could you start by filtering only .jpg files in your hash lists first, e.g.

grep -E '\.[jJ][pP][eE]?[gG]$' winXPimage1.txt >only-jpeg1.txt
grep -E '\.[jJ][pP][eE]?[gG]$' winXPimage2.txt >only-jpeg2.txt

Then get your list of md5s on the first system:

cut -f 1 -d ' ' only-jpeg1.txt > only-jpeg1-md5only.txt

Finally attempt your inverted search?

grep -v -f only-jpeg1-md5only.txt only-jpeg2.txt

Update: had to edit because my first two example lines had -v flags which I definitely didn't want. Well spotted by @Alex.