Fork and returning twice

Question

I am working on a project that requires implementation of a fork() in unix. I read freeBSD and openBSD source code but it is really hard to understand. Can someone please Explain the returning twice concept? I understand that one return is pid of a child, and that gets returned to parent and other one is zero and it gets returned to a child process. But I cannot wrap my head around how to implement this notion of returning twice... how can I return twice? Thanks everyone in advance.

Answer 1

When you call fork, it returns "twice" in that the fork spawns two processes, which each return.

So, if you're implementing fork, you have to create a second process without ending the first. Then the return-twice behavior will happen naturally: each of the two distinct processes will continue execution, only differing in the value they return (the child giving zero, and the parent giving the child's PID).

Answer 2

When you think of a function returning, you have your usual code flow in mind, which starts at the entry point (usually main) and then executes line by line, in a strictly deterministic and linear fashion.

However, in a real-world system, it is possible to have multiple execution contexts which each have their own control flow (and the new C++ standard actually includes that notion). Each separate process is an execution context that starts at main, but you can also create a new execution context from within an existing one (in fact, all operating systems must be able to do that!). fork is one way to create a new execution context, and the entry point of the new context is the point where fork returns. However, the original context also continues running, and it continues as usual after the fork call. The new context is a separate process, and thus fork returns (once) in both contexts.

There are other ways of creating new execution contexts; one is to create a new thread (within the same process) by instantiating a std::thread object or by using a platform-specific function; another is Linux's clone() function, which underlies both the Posix thread implementation and fork in Linux (by creating a new execution path for the kernel's scheduler, and either copying all virtual memory (new process) or not (new thread).

Answer 3

Following I will try to explain how to return twice from a function. I'm warning you from the start that this is all a hack. But there are plenty of places that use these sort of hacks.

First let's say we have the following C program.

#include <stdio.h>

uint64_t saved_ret;

int main(int argc, char *argv[])
{
        if (saveesp()) {
                printf("here! esp = %llX\n", saved_ret);
                jmpback();
        } else {
                printf("there! esp = %llX\n", saved_ret);
        }

        return 0;
}

Now we want to saveesp() to return twice so that we can reach both printf's. So here's how saveesp() is implemented:

#define _ENTRY(x) \
        .text; .globl x; .type x,@function; x:
#define NENTRY(y)       _ENTRY(y)

NENTRY(saveesp)
        movq    (%rsp), %rax
        movq    %rax, saved_ret

        movl    $1, %eax
        ret

NENTRY(jmpback)
        xorq    %rax, %rax
        pushq   saved_ret
        ret

This is in no way portable code. But you can write similar assembly stubs for all the platforms you want to support.

What saveesp() does is, it takes the return address stored on the stack and saves it to a local variable. Afterwards it returns 1. Which is a non-zero return, which takes us to the first printf.

After the printf() we call jmpback(). Which is the actual hack. This function makes it so that it appears that saveesp() returns a second time.

It does this by pushing the saved return address down the stack and doing a ret. The ret will pop the address from the stack and jump to it. The return code is set to zero this time around. So when we 'reach' back to our C routine it appears we've just came back from saveesp() with zero return value. Thus the second printf is reached.

If you're interested in this sort of hacks you should read a bit more about setjmp and longjmp from the C standard that are used to implement exception handling.

Also, we actually use this inside the OpenBSD kernel on the suspend/resume codepath. Have a look here at lines 231 and 250 it's pretty much the same C code as above. And then have a look at the assembly code here at line 542 is the savecpu function that returns the first time on suspend and at line 375 is where we return the second time around when we come back on resume.