Вопрос

I am writing a VSTi plugin, I need a coefficient calculating for part of the output envelope.

Basically I am receiving the incorrect answer to the mathematical expression, I have simplified it into several steps and it is still incorrect. The same expression works on a calculator, but it will not work translated into code. I have been using the debugger and checking the contents of the variables and have found that 0.25 / 88200 is giving a result of 2.8344671201814060e-006 and not 0.00000283446712 as it should be. Similar maths earlier in the code functions as expected.

currentDecCoeff = 0.25 / 88200;
is making currentDecCoeff = 2.8344671201814060e-006 (and not 0.00000283446712)

All relevant variables are of double type, I have even entered it as numeric values (as above) rather than variables, and it still gives the wrong answer.

Can anyone shed any light as to why this may be happening, and provide some ideas on things to try or even a solution.

I can add the code in context if needed. cheers

Ben

Это было полезно?

Решение 2

The notation e-006 means multiply what's left of me by 10^(-6). So indeed your result is correct.

Другие советы

They are the same to 9 significant figures, look up scientific notation.

To elaborate, scientific notation is used to show very small (or very large) numbers easily by removing leading or trailing zeroes. Typically, it has the following form:

y x 10^z

To get the value, solve the sum. C shows this as:

yEz

Thus, 2.4x10^6 (or 2.4E6) is 2400000.

Лицензировано под: CC-BY-SA с атрибуция
Не связан с StackOverflow
scroll top