Cryptography - Good to swip some bits in byte array?

Question

I currently informed myself about encryption a lot. And I wonder, whether it would be good to toggle some bits (XOR and bitmasks) at a known position in the encrypted byte array and to toggle them again before decrypting them.

Because even if you know the algorithm and the key it wouldn't be possible to decrypt them propably without knowing where to toggle the bits wouldn't it?

Answer 1

The bit-toggling becomes part of the algorithm, so if "they know the algorithm" comes to include which bits were toggled.

It does become marginally harder to find out that "the algorithm", but this gain is small. If they can get their hands on the key, I think your problem is somewhere else...

There are some disadvantages to this as well. First, you may introduce security flaws in the system. I don't think this will happen, but I don't know I won't, and in security you should assume you might cause security flaws unless you know you won't.

The second problem is that if you make a mistake here somewhere it is possible to corrupt data. Of course, rigid testing will make sure that a mistake won't make it to production, but it just isn't as safe as using the functionality of a security library.

Lastly, there is the problem that your code and data will be harder to work with. If you need to work with it in the future, or someone has to work with it, it'll probably take more effort than it otherwise would have.

Those aren't big things, but I'd say more than the gain. At the end of the day, this is little more than "security through obscurity", so no, I wouldn't say it is a good idea.

Answer 2

You need to look at Kerckhoff's Principle. Any crypto-system needs to be secure, even if the attacker knows the entire algorithm. Only the key must be kept secret. Your bit-twiddling is part of the algorithm, and so will be known to any enemy.

Consider AES. There are public papers describing in great detail exactly how AES works: NIST AES Description. That description of the algorithm is public, yet AES is still secure because if follows Kerckhoff. You need to make sure that your algorithm is secure. Simple bit-twiddling as you describe will not make an insecure algorithm secure. Any attacker will know what bits to twiddle and can untwiddle them and break the underlying insecure cypher.

As an alternative, you could add the bitmask you use to the key. This increases the key size and the processing time for very little security gain. There are usually better ways to increase the security of a cypher, such as adding more rounds.