Question

In our app, we currently live with the legacy of a decision to store all engineering data in our database in SI.

I worry that we may run the risk of not having sufficient precision and accuracy in our database or in .NET numeric types. I am also worried that we may see artifacts of floating-point maths (although that is probably a question all to itself).

For example, the source data may have been a pressure quantity expressed (and read in from some 3rd party service) in Psi (pounds per square inch). The engineers will have chosen this unit of measure because (for the quantity being expressed) this will tend to give easily-digested, human-readable numbers without requiring scientific notation.

When we 'standardise' the number, i.e. when we convert this quantity for our own persistence, we might convert it to Pa (Pascals) which will require either multiplying or dividing the number by some other potentially large number.

We often end up storing very large or very small numbers, and worse - we might do further calculations on these numbers.

At present we use ORACLE float and System.Double.

What do people think of this?

UPDATE

Further research has unearthed Units of Measure support in the forthcoming F# language (in CTP as I write).

It seems we'll be able to have F# understand user input such as:

9.81<n/s^2> // an acceleration

We'll also be able to create our own derived units and unit systems.

creating a derived unit for Newtons in F#
(source: msdn.com)

Was it helpful?

Solution

Keep significant figures in mind -- the accuracy of the measurement. If the PSI is known to only whole pounds, then after conversion to Pa there are 15 decimals, there is still only one significant figure.

Precision is different from accuracy, and performing floating point operations on engineering units need to take this into account during the operations - don't store more precision than the accuracy of the measurement, don't use more precision in a calculation than is known.


Edit:

You might also consider using NUMERIC(p,s) where the precision (number of digits) and scale (number of digits to the right of the decimal) can be explicitly specified.

If that is not an option, consider persisting the accuracy for a particular measurement so that it may be reported and/or used in calculations.

OTHER TIPS

I think as long as you're able to store exactly as much precision as you actually have, you have no reason to worry.

Using the example you gave of converting PSI to pascals (1 PSI = 6 894.75 pa), If I take a measurement of say 14.7 PSI and convert it to pascals I get 101,352.825. That's too much precision. You'd need to store that as 101,000 to reflect the real precision of the measurement, not the calculation.

Keep in mind that any numbers you use to do conversion need to be at least as precise as your measurements so you don't lose precision during conversion. It's best to have more digits of precision (at least one more) in your conversion factors than in your measurements.

I think engineering data usually isn't precise enough to worry about the difference. You know the engineer's expression "measure with a micrometer, mark it with chalk, cut with an axe". That about sums it up. Worrying about the difference between 8 significant figures or 12 in a calculation on something that is built in the real world to 2 significant figures tolerance just doesn't make sense.

To avoid loss of precision due to unit conversion, you can store all data that comes from measurement in the unit that it was measured in. Of course that means that you may end up with some pressure values being stored in Pa, others in Psi, or even mmHg. You have to decide yourself if that introduces more problems than it solves.

And I agree with the other answers: In most cases, the precision offered by an Oracle float is far higher than the precision of the measurement itself.

Well, it depends on how exact you want to be. Remember than when talking about engineering, it isn't enough to just store the number 3.20, because 3.2 isn't the same as 3.20 when it comes to engineering. 3.20 implies higher accuracy than 3.2, which could be 3.15 <= x < 3.25.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top