How to calculate double variable with 10 decimal precision in C++

Question

I'm going to calculate double type with 10 decimal point precision. of course, I hope the result precesion has same 10 decimal precision. However, it doesn't work, only 6 decimal point is possible in VS2008. Is there any reason or any idea ?

double dMag = 10;
double dPixelSize = 4.4;
double PixelWidth = 1024/2;
double PixelHeight = 768/2;
double umtomm = 1000;

double origin_PosX = 813.227696;
double origin_PosY = 676.195748;

double PosX = (origin_PosX - PixelWidth) * dPixelSize / dMag / umtomm;
double PosY = (origin_PosY - PixelHeight ) * dPixelSize / dMag / umtomm;

If I check PosX and PosY, then results are "0.132540, 0.128566" respectively. I expect that results are "0.13254018624000002, 0.12856612912" respectively.

thanks

Answer 1

From what I gather you are confusing a few things...

The precision of double is more than just 6 decimal places. I strongly recommend reading this wikipedia article which explains how the underlying double datatype stores your number and explains its precision.

The precision that is printed is limited to 6 decimal places by default. In order to fix that you need to learn how to use the format string.

In your comments you mentioned using

tmp1.Format("%f,%f\r\n", PosX, PosY); file.Write(tmp1,lstrlen(tmp1));

to print the strings.

Try changing this line to

tmp1.Format("%.10f,%.10f\r\n", PosX, PosY); file.Write(tmp1,lstrlen(tmp1));

and you will see that it will print 10 digits after the decimal point.