Password Hashes Characters in Applications

Question

I appreciate your time.

I'm trying to understand something. Why is it that hashes that I generate manually only seem to include alphanumeric characters 0-9, a-f, but all of the hashes used by our favorite applications seem to contain all of the letters [and capitalized ones at that]?

Example:

Manual hash using sha256:

# sha256sum <<< asdf
d1bc8d3ba4afc7e109612cb73acbdddac052c93025aa1f82942edabb7deb82a1  -

You never see any letters above f. And nothing is capitalized.

But if I create a SHA hash using htpasswd, it's got all the alphanumerics:

# htpasswd -snb test asdf
test:{SHA}PaVBVZkYqAjCQCu6UBL2xgsnZhw=

Same thing happens if you look at a password hash in a website CMS database for example. There must be some extra step I'm missing or the end format is different than the actual hash format. I thought it might be base64 encoded or something, but it did not seem to decode.

Can someone please explain what's happening behind the scenes here? My friend explained that piping "asdf" to sha256sum is showing the checksum, which is not the actual hash itself. Is that correct? If so, how can I see the actual hash?

Thank you so much in advanced!

Answer 1

There's two things going on here.

First, your manual hash is using a different algorithm than htpasswd. The -s flag causes htpasswd to use SHA1, not SHA256. Use sha1sum instead of sha256sum.

Second, the encoding of the hashes are different. Your manual hash is Hex encoded, the htpasswd hash is Base64 encoded. The htpasswd hash will decode, it just decodes to binary. If you try to print this binary it will look like =¥AU™¨Â@+ºPöÆ'f (depending on what character encoding you're using), and that may be why you believe it's not decoding.

If you convert the Base64 directly to Hex (you can use an online tool like this one), you'll find that sha1sum will generate the same hash.

My friend explained that piping "asdf" to sha256sum is showing the checksum, which is not the actual hash itself.

Your friend is incorrect. You're seeing the Hex encoding of the hash. But the piping does affect the hash that's generated, it adds a newline character, so what you're actually hashing is asdf\n. Use this command instead:

echo -n "asdf" | sha1sum

Answer 2

It is base64 encoded.

Base64 encoding ends an an equal sign. So that is the first indicator. Although the htpasswd man page doesn't mention it, other Apache docs about "the password encryption formats generated and understood by Apache" does say that the SHA format understood by Apache is base64 encoded.