PostgreSQL: pg_upgrade 9.5 to 13 failing with “connection to database failed: FATAL: database ”template1“ does not exist”

Question

I'm trying to upgrade a 9.5 server to 13 but the pg_upgrade process is failing after the main checks. Server is running CentOS 8 on EC2 and is using the pgdg repo installs of each server.

bash-4.4$ /usr/pgsql-13/bin/pg_upgrade --old-bindir /usr/pgsql-9.5/bin --new-bindir /usr/pgsql-13/bin --old-datadir /var/lib/pgsql/9.5/data --new-datadir /var/lib/pgsql/13/data --old-port=54320 --new-port=5432 --socketdir=/var/run/postgresql/
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Checking for tables WITH OIDS                               ok
Checking for invalid "sql_identifier" user columns          ok
Checking for invalid "unknown" user columns                 ok
Checking for roles starting with "pg_"                      ok
Creating dump of global objects                             ok
Creating dump of database schemas
                                                            ok

connection to database failed: FATAL:  database "template1" does not exist

could not connect to target postmaster started with the command:
"/usr/pgsql-13/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D "/var/lib/pgsql/13/data" -o "-p 5432 -b -c synchronous_commit=off -c fsync=off -c full_page_writes=off -c vacuum_defer_cleanup_age=0  -c listen_addresses='' -c unix_socket_permissions=0700 -c unix_socket_directories='/var/run/postgresql'" start
Failure, exiting

When I run that server start command manually, the server starts. There's nothing in the log that indicates what the issue is for the pg_upgrade command.

Due to the size of the databases in question I can't do a dump/restore as in my testing it took over 48 hours to perform that process even when copying on the same machine. So I'm hoping to get the upgrade process to work.

Answer 1

From the PostgreSQL mailing list:

The problem is that postgis, through gdal, ends up being linked to two different versions of proj at the same time.

You can check it by doing: ldd /usr/pgsql-13/lib/postgis_raster-3.so | grep proj

If that shows up two different proj libraries, then you have that same problem.

In this case, uninstall the OS supplied proj library. If that removes postgis through dependency, let it and then install it with:

dnf install --excludepkg proj --excludepkg proj-datumgrid postgis30_13