Understanding recursive factorial function in NASM Assembly

Question

Learning NASM Assembly on 32-bit Ubuntu. I'm now trying to learn about recursive functions, starting with factorial (note: here I am assuming that the parameter will always be non-negative).

Assuming that I have

push 3
call factorial

I want to end up with 6 in EAX.

Here is my attempt:

SECTION .text
global main
main:
; -----------------------------------------------------------
; Main
; -----------------------------------------------------------
push    3
call    factorial

; -----------------------------------------------------------
; Exit
; -----------------------------------------------------------
mov EAX,1
int 0x80

; -----------------------------------------------------------
; Recursive Factorial: n! = n * (n - 1)!
; -----------------------------------------------------------
factorial:

push    EBP         ; Retrieve parameter and put it
mov     EBP,ESP     ; into EBX register
add     EBP,8       ;
mov     EBX,[EBP]   ; EBX = Param

cmp     EBX,0       ; Check for base case
je      base        ; It is base if (n == 0)

dec     EBX         ; Decrement EBX to put it in the stack
push    EBX         ; Put (EBX - 1) in stack
inc     EBX         ; Restore EBX
call    factorial   ; Calculate factorial for (EBX - 1)
mul     EBX         ; EAX = (EAX * EBX) = (EBX - 1)! * EBX
pop     EBX         ; Retrieve EBX from the stack

jmp end
base:               ; Base case
mov     EAX,1       ; The result would be 1

end:

pop     EBP         ; Release EBP
ret

At least it works for the base case, ha... But for any other value I push, it always returns 0. I had the suspicion that maybe since EAX is 0, MUL would always result in 0, explaining this. To test, I decided to give EAX a value of 2, expecting some non-zero value, but it kept resulting in 0.

Can you advice me on how to do a recursive factorial function that takes its parameter from the stack? I believe having seen some examples, but either they were not recursive or they took their parameters from other places, or they used a bunch of variables (when I think it can be done with just registers).

Answer 1

Note that factorial(n-1) will overwrite factorial(n)'s value of EBX the first thing it does, thereby rendering the inc EBX after the push pointless. After you've reached the base case you'll have the situation where EBX is 0 when you do the mul, and of course anything * 0 == 0.

The easiest fix would be to change the prologue to:

push    EBP         ; Retrieve parameter and put it
push    EBX         ; save previous param
mov     EBP,ESP     ; into EBX register
add     EBP,12       ;
mov     EBX,[EBP]   ; EBX = Param

And the epilogue to:

pop     EBX         ; restore previous param
pop     EBP         ; Release EBP
ret