Mac OSX 64bit assembly - fork() test/jne doesn't work?

Question

I'm trying to write a program that performs the fork() syscall, and the child/parent each write a different string and exit(0).

The problem is - although fork() is successful (since I see 2 lines of output), for some reason both the parent and child process output the parent string.

Here's the code:

BITS 64

section .data
msg1:    db    "Hello from child", 0x0a
len1:    equ   $-msg1
msg2:    db    "Hello from parent", 0x0a
len2:    equ   $-msg2

section .text
global start

start:
        xor rax,rax
        mov rax,0x2000002    ; fork() syscall
        syscall

        test eax,eax
        jne flow_parent

        mov rax, 0x2000004   ; SYS_write
        mov rdi, 1           ; stdout
        mov rsi, msg1
        mov rdx, len1
        syscall

        xor rdi,rdi
        mov rax,0x2000001
        syscall

flow_parent:
        mov rax, 0x2000004   ; SYS_write
        mov rdi, 1           ; stdout
        mov rsi, msg2
        mov rdx, len2
        syscall

        xor rdi,rdi
        mov rax,0x2000001
        syscall

My output:

$ nasm -f macho64 code.s -o code.o
$ ld -o code -e start -macosx_version_min 10.7 code.o
$ ./code
Hello from parent
Hello from parent
$

Any ideas on how to fix this?

Answer 1

First off, you're making a syscall on Mac OS X with an assumption that it will behave just like the Linux kernel, which is clearly wrong (Linux fork syscall convention != XNU fork syscall convention). Mac OS X does not support users directly making system calls without going through the libSystem library.

Now leaving that aside, I'm not sure which version of the XNU kernel that you're running, but if you actually take a look at the source code for the fork function in the libsyscall library, you'll find that the edx register is used instead (orl %edx, %edx) to determine whether the process is the child or the parent.

Again, this would still not be reliable way of implementing this since Apple may easily change the interface in the future as they wish.

Answer 2

Don't test register eax! You need to test register rdx to get the return value!