Django Heroku Postgresql ERROR: invalid byte sequence for encoding "UTF8"

Question

I am using Django with Heroku Postgreql. I use both English and Turkish languages on default DB settings. My DB works with no issues both locally and on Heroku. However, I get an error when I try to dump and restore a db from local to production.

psql -U user -d db_name -f "b003.dump"

psql:b003.dump:270: ERROR:  invalid byte sequence for encoding "UTF8": 0xa3

Answer 1

First guess: Encoding mismatch

Your file b003.dump is not in the UTF-8 encoding.

You will need to override the encoding settings, specifying the correct text encoding for the file.

It's probably iso-8859-9 if it has Turkish text, though 0xa3 is the pound symbol (£) in many of the ISO-8859 encodings.

Try:

PGCLIENTENCODING="iso-8859-9" psql ....

I also suggest checking the output of the locale command to see what your system default encoding is, and of file -bi b003.dump to try to guess the encoding.

Second guess, after comments revealed file output

Your file isn't an SQL script style dump. It a PostgreSQL custom database dump. That's why file says:

b003.dump: PostgreSQL custom database dump - v1.12-0 

Restore it with the pg_restore command.

pg_restore -f b003.dump -d db_name -U user

At some point I want to enhance psql so it tells you this, rather than failing with an error.

The weird thing here is that you must've had a lot of previous errors, before the one you showed here, if it's a custom dump.

Answer 2

As pointed in the above answer the file is a postgres custom dump and can be restored by the command: pg_restore dump_file -d db_name -U user

http://www.postgresql.org/docs/9.2/static/app-pgrestore.html