The way you describe is basically as fast as you're going to get it. What's left would be any optimisations specific to your implementation of the solution, but without posting code, we cannot comment on that.
It could be worth comparing the trees that lead to the file instead of only the file to save a few allocations and marshalling costs; but you won't really get better algorithmically than comparing the tree entries.