Question

I am trying to analyze Linux assembly stack initialization/cleanup during function call/return using this code snippet below. The uninitialized variables are intended.

#define MAX 16

typedef struct _CONTEXT {
    int arr[MAX];
    int a;
    int b;
    int c;
};

void init(CONTEXT* ctx)
{
    memset(ctx->arr, 0, sizeof(ctx->arr[0]));
    ctx->a = 1;
}

void process(CONTEXT* ctx)
{
    int trash;
    int i;
    for (i = 0; i < MAX; i++)
    {
        trash = ctx->arr[i];
    }
}

int main(int argc, char *argv[])
{
    CONTEXT ctx;
    init(&ctx);
    process(&ctx);
    return 0;

}

As I learned from the school, and from this lecture slide,
an assembly of a function's stack initialization(style-1) should look like this:

pushq %rbp
movq %rsp, %rbp
subq $16, %rsp
movq %rdi, -8(%rbp)
...
leave
ret

But when I compile the code snippet above with gcc, the function main and init has the same stack initialization routine style-1 including subq instruction, to allocate stack variables' memory space,
but the function process does not have that kind of stack initialization.
I got this assembly code(style-2):

pushq %rbp
movq %rsp, %rbp
movq %rdi, -24(%rbp)
...
popq %rbp
ret 

So the questions are:

  1. What's the policy of compiler decision of creating different function stack initialization during compile time? I didn't put any __cdecl or such in this code, but 2 different stack initialization is found.

  2. How can I find out the allocated stack memory address and size when a style-2 function is initialized?

  3. What's the purpose of movq %rdi, -8(%rbp)?

  4. Are there more stack initialization styles beside style-1 and style-2 in Linux?
    (without mentioning __cdecl or __stdcall things explicitly)

Was it helpful?

Solution

The way functions are compiled are highly compiler specific and not OS specific. I'm sure that the code generated by GCC under 32-bit Windows is similar to the code generated by GCC under 32-bit Linux while the code generated by Sun's C compiler under 32-bit Linux will look different from the code generated by GCC. This is also the case for stack initialization! Therefore there can be MANY stack initialization styles depending on the compiler used, the compiler settings, the compiler version, internal compiler states and so on.

You are obviously running 64-bit code. Unlike 32-bit Windows (where __cdecl and __stdcall exist) in 64-bit Windows and in Linux there is only one calling convention: In 32-bit Linux this is equal to __cdecl in Windows; 64-bit Linux and 64-bit Windows use two different register-based calling conventions. This means: You cannot change the calling convention for Linux and 64-bit Windows programs because only one is supported.

The purpose of movq %rdi, -8(%rbp) is to store the argument (in the rdi register) on the stack; movq %rdi, -24(%rbp) does the same but it is writing to some area of the stack that may be overwritten by signal handlers - not a good idea! However this is no problem if the value is not read back from the stack!

Obviously the "style-2" function does not require any stack memory.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top