Simplest way to remove something from the FPU stack

Question

I've been having some trouble lately with FPU stack overflows. I managed to track it back to a buggy library function that pushes a garbage value onto the FPU stack every time it's called and never cleans it up.

Fortunately, this is easily reproducible and I know exactly what conditions cause it. I can drop a block of inline ASM into the routine that calls this routine to pop the top value back off the FPU stack... except I don't quite know what to write. My ASM-fu is fair to middlin', but not that strong.

So what's the simplest way to get rid of the top value on the FPU stack in x86 assembly, assuming it's garbage data and I don't care about the value?

Answer 1

If you know how much you need to adjust the stack by, you can use fincstp. You also want to ffree the registers that you increment over.

However, probably the simplest solution is to use one of the popping data transfer operations like fstp. Normally you would store the result into an area of memory for later use, something like:

mem_area: defs 10         ; ten bytes for 80 bits
          fstp mem_area   ; pop it

But, if you know you just want to throw away the value, you can use st(0) itself as the destination, saving the memory requirement:

fstp st(0)

See here for a detailed guide on the instructions (particularly this bit).

Answer 2

For Delphi/BASM, in my view the simplest way to pop the FPU stack once is:

asm
 fstp st(0)
end;

Answer 3

If st0 is the only x87 register in use, you can empty it with:

ffree st0

But this is different from a normal pop if there are multiple stack registers in use, because it doesn't adjust the top-of-stack pointer (TOP field in the x87 status word).

See the registers chapter of the Simply FPU x87 tutorial.

st1 would still be st1 after freeing st0 instead of popping, so this is normally not what you want and has no significant advantage over fstp st0.

Answer 4

just pop it off the stack with any (fast) instruction that pops. 8087 instruction set

if that doesn't work, FUCOMPP pops twice.