Duplicate ids returned from python appengine cloud endpoint

Question

I have written a very simple application on app engine using the endpoints-proto-datastore library. I'm having trouble with my list method. I need to query the data from a javascript client and be able to update any particular entity based on user input. The endpoint is returning an array of entities as expected, but sometimes there are two entities which share the same id. This makes it impossible to reliably update the entity in the datastore as I can't tell which one the id actually belongs to.

Here, a screenshot from the datastore viewer tool:

and the result from calling the list method through the api explorer:

200 OK
<headers omitted>
{
 "items": [
  {
   "id": 5906470911296406000,
   "prices": [
    "$1000.00"
   ],
   "options": [
    "Chrome"
   ],
   "title": "New Equipment",
   "quantity": true
  },
  {
   "id": 5906470911296406000,
   "title": "New Equipment",
   "quantity": false
  }
 ]
}

You can see the ids are duplicated in the api explorer but not in the datastore. So far I have been unable to reliably produce this behavior, but it seems to happen only when I add two entities to the datastore which are very similar as they are above.

My Model:

class AvailableEquipment(EndpointsModel):
    _message_fields_schema = ('id', 'title', 'options', 'prices', 'quantity')
    title = ndb.StringProperty()
    options = ndb.StringProperty(repeated=True)
    prices  = ndb.StringProperty(repeated=True)
    quantity = ndb.BooleanProperty()

My api:

@endpoints.api(name='equipment', version='v1', description='API for available equipment data')
class AvailableEquipmentAPI(remote.Service):

    @AvailableEquipment.method(path='equipment', http_method='POST', name='insert')
    def EquipmentInsert(self, equipment):
        equipment.put()
        return equipment

    @AvailableEquipment.query_method(path='equipment', name='list')
    def EquipmentList(self, query):
        return query

All of these tests have been carried out on the local development server. Thanks for any help you can give.

Answer 1

It's due to the fact that Javascripts stores the numbers using 64-bit floats.

They already changed that and actually in the version 1.8.0 they fixed it and the auto ID is smaller so they can be represented in JSON. In the latest blog post they announced that in the upcoming version this will be guaranteed and the scattered auto IDs will be by default.

Answer 2

Are you using the most recent SDK? A previous version was generating IDs that were too large for a javascript float to store, so two unique IDs could round/truncate to the same value. That your IDs end in 000 suggest this might be the cause. The latest version, and the live environment, don't suffer this problem.