Querying for floating point values in SQLAlchemy

Question

To summarize the problem, I can not select any entity against a floating point value using SQLAlchemy.

For example:

m = session.query(Model).get(1);
all_m = session.query(Model).filter(Model.some_float_value, m.some_float_value)

all_m is empty while I would expect it to always have at least one!

How can I filter a float value in SQLAlchemy, based on an arbitrary precision (EG. some I may want to match to 0.01 or others I may filter with a precision of 0.0005).

For example, I want to be able to write a generic function so I can write queries like this in my code:

session.query(Model)\
    .filter(Model.foo == "bar",
            match_float(Model.some_float_value, float_val, 0.025)).all()

Where there tolerance of the matches anything within an approximate tolerance of 0.025.

However, I am not very familiar with SQLAlchemy, and was unable to locate documentation on how to create custom compare function, or any built in functions that would do what I need.

Is there a built in function I can use, a way to provide a custom compare function for the filter method, or do I a way to write a raw query/procedure for this?

Aditional Info:

Postgres 9.2

SQLAlchemy 0.9

Values are stored as double precision floating point

All Models were defined like so (there are a lot of them, which makes a solution with explicit column definitions less than desirable):

engine = create_engine('postgresql://user:pass@localhost:1234/database')
Base = declarative_base()
metadata = MetaData(bind=engine)

class Model(Base):
    __table__ = Table('model', metadata, autoload=True)

edit

Iv'e completely reworded my question for clarity. Sorry for the confusion, I fail at communicating with humans...

Answer 1

One concrete example for custom comparisons is shown in GeoAlchemy (though it may be overkill for your application). While the examples in the link below use bounding boxes for comparing spatial coordinates, you could use a simpler approach to implement comparison within a tolerance.

See for example the Comparator class which defines "~=" and other operators here:

http://geoalchemy-2.readthedocs.org/en/0.2.4/spatial_operators.html

Since you mention that you were unable to find documentation on how to implement a custom compare function, see here:

http://docs.sqlalchemy.org/en/latest/core/types.html#types-operators

On a related note - if you want to prevent database updates from being triggered if a floating point column remains within some threshold, see the discussion here:

https://groups.google.com/forum/#!msg/sqlalchemy/tCpsGZmjk_w/lOwW93qHV0sJ

(Disclaimer: This is based on a google search, I am new to SQLAlchemy)

Answer 2

One of the rules of thumb for floating point numbers is "Never compare floating point numbers for equality." (You can Google that quote.)

The simplest and most predictable way forward is to use a SQL decimal(n,m) data type instead of a float. The SQL decimal(n,m) data type is free from errors of approximation.

SQLAlchemy supports SQL decimals.

If you must operate with doubles, your best bet is probably to follow one of the two methods recommend by the C FAQ.

One problem with calculating the inequality (abs(a-b) >= 0.01) is that the result will be calculated using the closest floating-point approximations to a and b. Those floating-point approximations might be exactly the values you expect them to be, but that's application-dependent. (The decimal values 45.06 and 45.062 are not what you expect them to be when they're assigned to a double. In other words, the closest floating-point approximation to 45.06 isn't 45.06.)

The C FAQ has something to say about using this kind of expression as a general-purpose way of determining equality.