To Xor two byte arrays simply use a for loop and the Xor operator.
VB.Net's Xor will compile to a the CIL Xor opcode which should be subsequently JIT compiled to the very fast x86 XOR processor instruction.
The cost of the Xor operation is likely to be negligible in comparison to the cost of file I/O.