How to remove duplicate entries of an array by checking the entries of another array in ruby?

Question

I am trying to implement a tail like program in Ruby. However, my tail method returns an array as number of lines(which are limited, i.e. 10 for this example, which means the array size with always be 10) it reads.

So, for the first time, I save all last 10 lines of the log file I read. Which may look something like this:

["\n", "Some Data\n", "\n", "One Value here\n", "\n", "sure sure\n", "\n", "Random thoughts\n", "\n", "Wait!!\n"]

One I modify the log file and append some new lines with new data. My tail returns a new array(with same limited size, i.e. 10). Which may look like this:

["\n", "sure sure\n", "\n", "Random thoughts\n", "\n", "Wait!!\n", "\n", "Error - nothing found\n", "\n", "go to sleep\n"]

So, if you notice the second array shows the progress. See the value at index 5 of first array is now at index 1 of the second array. So, this makes an obvious choice of displaying all the contents or changes happened after "Wait!!\n" entry of second array. Now, I am facing this issue with regards to avoiding the repetition/duplicate values.

So, if I take first array: ar1 and second array: ar2 and do: ar2 - ar1, it shows:

["Error - nothing found\n", "go to sleep\n"]

Which is not correct, as ideally it should have been this:

["\n", "Error - nothing found\n", "\n", "go to sleep\n"]

Basically, whatever came after the last line. I just need the changed lines, and want to remove only those duplicates which occurred in previous tail output. Is there anyway to do it?

Answer 1

ary1 = ["\n", "Some Data\n", "\n", "One Value here\n", "\n", "sure sure\n", "\n", "Random thoughts\n", "\n", "Wait!!\n"];
ary2 = ["\n", "sure sure\n", "\n", "Random thoughts\n", "\n", "Wait!!\n", "\n", "Error - nothing found\n", "\n", "go to sleep\n"];
ary2[(ary2.index(ary1.last) + 1)..(ary2.size - 1)]
=> ["\n", "Error - nothing found\n", "\n", "go to sleep\n"]

Answer 2

Rather than reinventing the wheel and trying to do complicated heuristic de-duplication of arrays, I recommend using a library like File::Tail which implements tail-like behavior in Ruby.

https://github.com/flori/file-tail

Answer 3

You actually don't want to remove duplicates, you want to only show those things that you haven't yet shown.

Take the value of the last item in ar1, find the index of that item in ar2, print the tail of ar2. ( code outline w/o error handling )

shown = ar1.last
show = ar2.index(shown)
print ar2[show..-1]

However, that actually doesn't cover all possible cases. The problem is that an array is a really poor match to the underlying data structure.