How to convert PostgreSQL escape bytea to hex bytea?

Question

I got the answer to check for one certain BOM in a PostgreSQL text column. What I really like to do would be to have something more general, i.e. something like

select decode(replace(textColumn, '\\', '\\\\'), 'escape') from tableXY;

The result of a UTF8 BOM is:

\357\273\277

Which is octal bytea and can be converted by switching the output of bytea in pgadmin:

update pg_settings set setting = 'hex' WHERE name = 'bytea_output';
select '\357\273\277'::bytea

The result is:

\xefbbbf

What I would like to have is this result as one query, e.g.

update pg_settings set setting = 'hex' WHERE name = 'bytea_output';
select decode(replace(textColumn, '\\', '\\\\'), 'escape') from tableXY;

But that doesn't work. The result is empty, probably because the decode cannot handle hex output.

Answer 1

If the final purpose is to get the hexadecimal representation of all the bytes that constitute the strings in textColumn, this can be done with:

SELECT encode(convert_to(textColumn, 'UTF-8'), 'hex') from tableXY;

It does not depend on bytea_output. BTW, this setting plays a role only at the final stage of a query, when a result column is of type bytea and has to be returned in text format to the client (which is the most common case, and what pgAdmin does). It's a matter of representation, the actual values represented (the series of bytes) are identical.

In the query above, the result is of type text, so this is irrelevant anyway.

I think that your query with decode(..., 'escape') can't work because the argument is supposed to be encoded in escape format and it's not, per comments it's normal xml strings.

Answer 2

With the great help of Daniel-Vérité I use this general query now to check for all kind of BOM or unicode char problems:

select encode(textColumn::bytea, 'hex'), * from tableXY;

I had problem with pgAdmin and too long columns, as they had no result. I used that query for pgAdmin:

select encode(substr(textColumn,1,100)::bytea, 'hex'), * from tableXY;

Thanks Daniel!