convert c++ int to VARBINARY and back again

Question

I've just discovered that all but VAR data types do not vary their LENGTHs. This strikes me as extremely inefficient since there may be INT fields like an autoincrement PRIMARY that never change where a 1 in a BIGINT column will take the full 8 bytes.

I have many columns that won't change once they're inserted but have the potential to be large. Instead of using all BIGINT columns, I'd like to use VARBINARY instead.

I want to manipulate the data as an int in c++ yet store it as VARBINARY in mysql.

How can these conversions be made in c++?

Answer 1

I would highly recommend against this.

If you look at the MySQL documentation for VARCHAR, you'll see that

VARCHAR values are stored as a 1-byte or 2-byte length prefix plus data. The length prefix indicates the number of bytes in the value.

Let's take a 64-bit value like 0xDEADBEEF2B84F00D. If you store that in a BIGINT field, it takes 8 bytes (it's stored as a binary value.)

If you store that as a VARCHAR, even if you assume the numbers are always hexadecimal, we're talking 1 or 2 bytes for the length field, and then another 16 bytes for the hexadecimal characters! 18 bytes, and the performance lost by having to convert to/from ASCII, this is a terrible trade-off!

Let's look at the opposite case, 0x01. Stored in a BIGINT field, it's still 8 bytes. Using the same convention from before, it's probably going to be 3 bytes in your database. Not even close to being worth it, in my opinion. Especially when you consider how poorly the database would perform with a bunch of ASCII data.

Just don't do it. Use the database the way it was designed.

Answer 2

I think this is a losing proposition. Unless you can show that your data simply will not fit on your system without resorting to kludges, I think you're better off sticking with integer types for integer values. Otherwise, what's to prevent someone from inserting "hello" in your numeric ID column, and how happy will you be querying numbers that don't look like numbers? It'll probably be slower overall too.