Found the solution to my problem, the ^ is a bitflip. Thus if you have a bitmask and use the xor operator on the mask, you flip the bit on that place. E.g. 1010 ^ (1<<1) results in 1000. Same goes for 1000 ^ (1<<1) = 1010.
The substraction also works, but with the xor operator you know for certain that you only touch the bit at that place, and none else. Image 1000 - (1<1), thus would result in something entirely different. Thus substraction works and can be used if you are 100% sure that at an 1 is at place i, but xor is safer.