This is not safe in general. It's programs that pull tricks like this that led to the /LARGEADDRESSAWARE
flag in Windows. (Aka "this is why we can't have nice things.")
What you can do is use the bottom bits instead of the top bits. Since your structure contains a pointer, it is already 8-byte aligned, which means that the bottom three bits are always 0, so you can use those bits as tag bits.