3GPP TS 27.007 Section 5.5 gives the allowed coding schemes supported by the AT+CSCS command. As far as I am aware, none of them are 6 bit. But it does say that
(conversion schemes not listed here can be defined by manufacturers)
So it seems that as you suspected, you may have a user-defined alphabet. How to deal with this?
If you can't get any information from the sender, then I would try fitting your messages to the first 6 bits of all of the officially supported conversion alphabets, to see if you can match all the characters.
Consider the contents of the messages - are they a limited character set, e.g. all numbers (0-9)?
It is possible that the sender may have used an official alphabet, maybe one that contains all the characters that they need in the first 6 bits, so they did not need to use the 7th and 8th bit.