delete_node failure with osm2pgsql

Question

I updated to OS X Mavericks and I try to setup again a workflow to convert personnal OSM data (created using JOSM software) into Tilemill maps.

For this, I use osm2pgsql to populate a postgres/postgis database with my OSM files. Before update, the same workflow worked well.

I use Postgresql.app version 9.3.0.0 and osm2pgsql version 0.84.0

When I launch the osm2pgsql command, I get this error :

osm2pgsql SVN version 0.84.0 (64bit id space)

Using projection SRS 900913 (Spherical Mercator)
Setting up table: coast_point
...
Reading in file: ../src/misc/00_Coast.osm
delete_node failed: ERROR:  prepared statement "delete_node" does not exist
(7)
Arguments were: -476852, 
Error occurred, cleaning up

So, there is a "delete_node" error, and I really don't know why. I tried to change the negative 'id' values to positive ones, but I have the same error.

Here is the first line of the OSM file that caused the error :

<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' upload='true' generator='JOSM'>
  <node id='-476852' action='modify' visible='true' lat='-4.660264310091712' lon='11.79785544887142' />
  <node id='-476850' action='modify' visible='true' lat='-4.659760277426281' lon='11.78306037634432' />
...

Same error on all files that worked previously.

I opened a bug report on osm2pgsql github but this forum is not very active, so I don't expect any help from there.

I've found in osm2pgsql code that the delete_node part is in osm2pgsql/middle-pgsql.c file :

           "PREPARE get_node (" POSTGRES_OSMID_TYPE ") AS SELECT lat,lon,tags FROM %p_nodes WHERE id = $1 LIMIT 1;\n"
           "PREPARE get_node_list(" POSTGRES_OSMID_TYPE "[]) AS SELECT id, lat, lon FROM %p_nodes WHERE id = ANY($1::" POSTGRES_OSMID_TYPE "[])",
           "PREPARE delete_node (" POSTGRES_OSMID_TYPE ") AS DELETE FROM %p_nodes WHERE id = $1;\n",
     .copy = "COPY %p_nodes FROM STDIN;\n",
  .analyze = "ANALYZE %p_nodes;\n",
     .stop = "COMMIT;\n"

  (...)
  pgsql_execPrepared(node_table->sql_conn, "delete_node", 1, paramValues, PGRES_COMMAND_OK );

If you have any idea, you're very welcome !

Thanks

Greg

Answer 1

Helped by osm2pgsql guys, I figured out that the problem was mainly due to the use of JOSM files into osm2pgsql.

In fact, JOSM files are not pure OSM files as there are some missing key/values : version, user and timestamp.

As I don't need those tags, I preprocessed the OSM files from josm with this script in order to pass the compatibility tests :

#!/bin/bash

SOURCE=$1
TARGET=$2

cat $SOURCE | sed s/"node id=\'-"/"node id=\'"/g | sed s/"nd ref=\'-"/"nd ref=\'"/g \
    | sed s/" action=\'modify\'"//g \
    | sed "/node/ s/ timestamp='[^']*'//" \
    | sed "/node/ s/ action='[^']*'//" \
    | sed "/node/ s/ version='[^']*'//" \
    | sed "/node/ s/ user='[^']*'//" \
    | sed "/node/ s/ id/ version='1' user='iero' timestamp='1970-01-01T12:00:00Z' id/" \
    | sed "/way/ s/ timestamp='[^']*'//" \
    | sed "/way/ s/ action='[^']*'//" \
    | sed "/way/ s/ version='[^']*'//" \
    | sed "/way/ s/ user='[^']*'//" \
    | sed "/way/ s/ id/ version='1' user='iero' timestamp='1970-01-01T12:00:00Z' id/" \
    | sed "/relation/ s/ timestamp='[^']*'//" \
    | sed "/relation/ s/ action='[^']*'//" \
    | sed "/relation/ s/ version='[^']*'//" \
    | sed "/relation/ s/ user='[^']*'//" \
    | sed "/relation/ s/ id/ version='1' user='iero' timestamp='1970-01-01T12:00:00Z' id/" \
    > $TARGET

It's not the most beautiful/optimal script we can make, but it seems to works well. I have my data in the pgsql database now.

With this script, I might be able to pass Osmosis tests too !

Thanks to you all

Greg