Assembly FPU Stack Breaking; Resulting in -1.#IND

Question

I'm newish to assembly in general, and especially to usage of the FPU.

I'm writing an assignment for school that calculates the Standard Deviation of an array of random numbers.

The array is loaded and the mean calculation works fine. The entire process works for an array of 7 values or smaller, but returns -1.#IND (This means NaN?) for an array size of 8 or more elements.

Something is happening within the loop that sums the (mean - element)^2 values. I'm not sure what's happening, but I'm assuming the FPU stack is breaking in some way.

If anyone can point me in the right direction, I'd be thoroughly pleased.

fla = REAL8

; STD DEVIATION
stdDev:
        call    meanCalcFunc            ; fmean = loaded

        mov     ebx, offset array1      ; Location of Element into EBX
        mov     ecx, [esp+4]            ; ECX = num of elements in array
        mov     mem1, 0                 ; Mem1 = 0
        fld     mem1                    ; ST = 0
        fstp    fla                     ; fla = ST = 0

mainFunc:
        mov     eax, [ebx]              ; Array Element into EAX
        mov     mem1, eax               ; Array Element into mem1
        fld     fmean                   ; ST = mean of array

        fisub   mem1                    ; mean - element -> ST

        fld     ST                      ; Now ST and ST(1) = difference of mean - element
        fmulp   ST(1), ST               ; Difference^2 in ST(1)

        fld     ST(1)                   ; Copy Difference^2 to ST

        fadd    fla                     ; FLA += Difference^2
        fstp    fla                 

        add     ebx, 4                  ; Move to next element in array
        loop    mainFunc

        fld     fla                     ; ST = Sum of squares

        mov     ecx, [esp+4]            ; Num of elements to ECX
        mov     mem1, ecx               ; Num of elements to mem1

        fidiv   mem1                    ; ST = Sum of squares / num of elements

        fst     fla

        fsqrt                           ; ST = final result

        fstp    fla                     ; flb = final result

        INVOKE _gcvt, fla, 8, ADDR outstring
        output  devlbl, outstring

        ret

Answer 1

Your loop contains an unbalanced number of loads and pops, so each loop iteration leaves one more item on the x87 stack. When the stack overflows, you get the result you are observing. Don't do that.

instruction        elements on stack
-------------------------------------------
mainFunc:          k
fld     fmean      k + 1
fisub   mem1       k + 1
fld     ST         k + 2
fmulp   ST(1), ST  k + 1
fld     ST(1)      k + 2
fadd    fla        k + 2
fstp    fla        k + 1
loop    mainFunc   k + 1