Compare 2 files NOT line by line in shell

Question

I need to compare two files NOT line by line ,rule out common lines,then have an output with everything that is different using shell commands .

Example:

file 1:

5
124
346
12
65
8
78

file 2:

10
23
129494
5
493
124
4999
346

Output:

12
65
8
78
129494
943
4999

Thank you

Ok let me add some details: I have some files including pairs of IPs.

Example:

file 1:

55.4.56.11 10.22.123.43 10.22.123.43 147.34.123.43 147.34.23.2 23.124.251.1

file 2:

123.4.23.89 121.45.60.0 121.45.60.0 0.0.0.0 120.3.2.129 45.55.68.09 45.55.68.09 66.67.23.111 55.4.56.11 10.22.123.43

So in this example,i need as output,every line of both files except : 55.4.56.11 10.22.123.43

That means i cant use numerical comparison.Also sorting out the files doesn't help,cause the may have different number of rows.I need something like "global" comparison for both files. If you guys need more details,i will gladly edit my post further. Thank you for your time. (I cant make 2nd example look like the first,i dunno why,but suppose there is a newline after every two IPS)

Answer 1

Using grep:

grep -xv -f f2 f1 && grep -xv -f f1 f2
12
65
8
78
10
23
129494
493
4999

Answer 2

This is a good candidate for the comm command and process substitution.

comm -3 <(sort -un f1) <(sort -un f2)

If you want a flat output file you will want to trim the leading whitespace output by the comm command

comm -3 <(sort -un f1) <(sort -un f2) | tr -d '\t'

Answer 3

Alternative solution with awk:

awk 'BEGIN { while ( getline < "f2.txt" > 0 ) _[$1]++ }{if (!($1 in _)) {print $1}}' f1.txt && awk 'BEGIN { while ( getline < "f1.txt" > 0 ) _[$1]++ }{if (!($1 in _)) {print $1}}' f2.txt

Details:

awk 'Command1 Command2' fileName

Command1: BEGIN { while ( getline < "f2.txt" > 0 ) _[$1]++ } Reads each line in f2.txt and stores into a data structure.

Command2: {if (!($1 in _)) {print $1}} For each line in file f1.txt processed by awk, prints line that's not present in f2.txt

fileName: f1.txt file processed by awk.

This is applied on each file, and we have the result.

Answer 4

Using java :

   import java.io.BufferedReader;
    import java.io.File;
    import java.io.FileReader;
    import java.io.IOException;
    import java.io.InputStreamReader;
    import java.util.HashSet;
    import java.util.Set;

    public class ComapreServerPackage {

        public static void main(String[] args) throws IOException {
            BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));

            System.out.println("Path of source file : ");
            String sourceFilePath = reader.readLine();

            System.out.println("Path of target file : ");
            String targetFilePath = reader.readLine();

            File sourceFile = new File(sourceFilePath);
            File taregtFile = new File(targetFilePath);

            BufferedReader fileReader = new BufferedReader(new FileReader(sourceFile));

            String readLine = "";

            System.out.println("Comparing ...");


            Set <String> sourceSet = new HashSet();

            // file source set 
            while ((readLine = fileReader.readLine()) != null) {
                sourceSet.add(readLine);
            }

            fileReader.close();
            fileReader =  new BufferedReader(new FileReader(taregtFile));

            System.out.println("Lines not available in source file are : ");
            // file source set 
            while ((readLine = fileReader.readLine()) != null) {
                if(!sourceSet.contains(readLine))
                    System.out.println(readLine);
            }

            System.out.println("*************** end *********************");
            reader.close();
            fileReader.close();

        }

    }