Handling numbers that are prices with GAE + WTForms

Question

I use Google app engine (python) and Jinja2 and I'd like to take advantage of webapp2's feature for currency internationalization and localization ie store prices as a number and make the correct formatting using the builtin i18n method of webapp2. Then first I must have the correct data type for my prices.

I'm migrating from a prototype that used String for prices to a more real implementation where I use some number data type for prices + another variable which currency it is. So in the migration meantime I will use both variables, make the migration to using numbers instaed of strings, remove the string datatype and make a mapreduce job to update the old entities to a number-based variable such as integer or decimal instead of allowing a text / a string to represent a price.

My environment (google app engine) doesn't support the decimal data type and I might as well use integers only since the usage is buying and selling more exapensive articles and objects such as houses, cars, etc. where dollars matters and cents usually are listed only as 00. So I dropped the project of implementing a decimal property and it was bugging anyway and now I ask which is the correct way to set the price and if I must cast it before it is save?

Ie. is this correct:

form = EntityForm(self.request.params)
if form.validate():
    entity.title = form.title.data
    entity.name = form.name.data
    entity.email = form.email.data
    entity.text = form.text.data
    entity.set_password(form.password.data)
    entity.price = form.price.data
    try:
      if form.price.data:
        form.price.data=form.price.data.replace(',','.').replace(' ', '')
        entity.integer_price = int(form.price.data)
    except:
      pass

Or do I not have to cast the variable to an integer and type conversion will be taken care of behind the scenes? I believe the form variables are strings and price must be a number to get sorted and filtered properly.

When I was using I custom-made decimal property I started getting this error:

Traceback (most recent call last):
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~montaoproject/validation.355292363398040911/i18n.py", line 653, in get
    ads = paginator.page(page)
  File "/base/data/home/apps/s~montaoproject/validation.355292363398040911/paginator.py", line 42, in page
    return Page(self.object_list[bottom:top], number, self)
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/db/__init__.py", line 2189, in __getitem__
    return self.fetch(limit, start)
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/db/__init__.py", line 2084, in fetch
    return list(self.run(limit=limit, offset=offset, **kwargs))
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/db/__init__.py", line 2237, in next
    return self.__model_class.from_entity(self.__iterator.next())
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/search/__init__.py", line 463, in from_entity
    return super(SearchableModel, cls).from_entity(entity)
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/db/__init__.py", line 1411, in from_entity
    entity_values = cls._load_entity_values(entity)
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/db/__init__.py", line 1387, in _load_entity_values
    value = prop.make_value_from_datastore(entity[prop.name])
  File "/base/data/home/apps/s~montaoproject/validation.355292363398040911/main.py", line 98, in make_value_from_datastore
    return decimal.Decimal(value)
  File "/base/python27_runtime/python27_dist/lib/python2.7/decimal.py", line 548, in __new__
    "Invalid literal for Decimal: %r" % value)
  File "/base/python27_runtime/python27_dist/lib/python2.7/decimal.py", line 3844, in _raise_error
    raise error(explanation)
InvalidOperation: Invalid literal for Decimal: u''

The code I was using was:

class DecimalProperty(db.Property):
  data_type = decimal.Decimal

  def get_value_for_datastore(self, model_instance):
    return str(super(DecimalProperty, self).get_value_for_datastore(model_instance))

  def make_value_from_datastore(self, value):
    return decimal.Decimal(value)

  def validate(self, value):
    value = super(DecimalProperty, self).validate(value)
    if value is None or isinstance(value, decimal.Decimal):
      return value
    elif isinstance(value, basestring):
      return decimal.Decimal(value)
    raise db.BadValueError("Property %s must be a Decimal or string." % self.name)

But this didn't work for me, I started getting the word "None" as a string for articles / object that had no pricing which made my app not load properly so I removed the decimal attempt completely and I'm going for the solution with integerproperty instead and validte that input either is empty or an integer which I can do with wtforms.

So I'm not using the custom decimal anymore nd we decided that integerproperty will do and worst case if fraction of a currency is needed then a can use another variable "cents" so that I use integers for dollars and integers for cents and the latter is rarely or never used.

So the solution I'm planning now is simple:

class Article(GeoModel, search.SearchableModel):

integer_price = IntegerProperty() #store dollars as dollars
currency = db.StringProperty(choices=(
    'EUR',
    'ARS',
    'AUD',
    'BRL',
    'GBP',
    'CAD',
    'CZK',
    'DKK',
    'HKD',
    'HUF',
    'ILS',
    'INR',
    'JPY',
    'MXN',
    'NZD',
    'NOK',
    'PLN',
    'PHP',
    'SGD',
    'SEK',
    'SGD',
    'CHF',
    'USD',
    'THB',
    'TWB',
    ), verbose_name='Currency')
price = db.StringProperty(verbose_name='price')  #quit using this and use a number instead

Do you know if I actually must make the type conversion or any other pitfall that I might fall into? Do you agree that I don't need decimals for my use case (advertising a car, advertising a house) etc though it's conceivable with more exotic pricing that I have to disallow eg. "20 dollars per gallon" is a price that I can't handle but it can be specified in the text instead since it's a special type of price (per unit volume) and likewise if the price is in cents. Storing price as integerproperty will allow for much better formatting of prices expecially since my application is internationalized and localized.

Thanks for any answer or comment

Answer 1

Please have a look at the issue number 6595 and also this. If your problem lies in this area it could be solved by the patch else I would suggest to capture the area in an except block and dump the variable esp the individual ordinal number's to make sure, what is passed to the Decimal is a legitimate acceptable character. To me it look's a bit strange that the character its complaining about is an empty string so let's try to confirm that all the character's are valid as we are dealing with unicode here.