Question

I understand that you can handle integer overflow by using compiler flags and signal handling (see accepted answer to this question).
In fact, you can actually prevent it.

However I've been unable to find any information on how to handle/avoid stack overflows:

void breakStack(int num){
     // Implied check to see if addition will cause integer overflow
     int tmp = num + 1;
     // Code to see if next call will cause a stack
     // overflow and avoid it
     breakStack(tmp);
}

Or:

void breakStack(int num){
    // Implied check to see if addition will cause integer overflow
    int tmp = num + 1;
    breakStack(tmp);
}

void stackOverFlowHandler(int signum){
    // Code to handle stack overflow
}

Two of the solutions I've found are: "Do not use recursion" and "Compiler optimizations can remove tail recursion, therefore the problem is solved". But I'm looking for a different solution.

EDIT: I fond a way to avoid it (see accepted answer).

Avoid solution:
Say you have the size of the function (e.g. 50 bytes), and the amount of free space left on the stack (e.g. 200 bytes), then:

void tryToBreakStack(int num){
    // Implied check to see if addition will cause integer overflow
    int tmp = num + 1;
    if(functionSize>leftSpace){
        return;
    }
    tryToBreakStack(tmp);
}

Now, I might be able to get the function size with nm -S and hard code it (I know the accepted answer says it can't be done). Additional note: I'm almost sure this value can't be obtained by calling sizeof(tryToBreak) or sizeof(&tryToBreak).
In addition, I might be able to set the stack size and get a rough approximation of free space on the stack.

Before I start working on that solution (and maybe see it failing), I'd like to know if there is another way of doing it (perhaps easier and more precise).

EDIT: Now that I found a way of avoiding it, the real question is: How do you recover from a stack overflow using signal handling.

Handling solution:
I'm not sure what to do when the process receives a "segmentation fault" signal and it has to handle it. First: how do you know it was caused by a stack overflow?, and second: what do you to recover from a stack overflow?

Was it helpful?

Solution

It's possible to define your own stack, although it's hard to do in a completely portable fashion. In principle, you can do it with posix threads (see pthread_set_stackaddr) so its not even particularly exotic.

Then you can use mprotect to create a red zone ("guard area") at the end of your stack. Stack overflow will then result in a SIGSEGV, which you can handle; the sigaction handler will be provided with the address of the protection fault, so you can check in the handler to see if its part of one of your red zones. (Of course, you have to run the signal handler with an alternate stack. See the SA_ONSTACK flag and sigaltstack.

Recovering from stack overflow (without compiler assistance) is trickier.

By the way, there is no relationship between the size of a function (as a relocatable object consisting of a series of machine instructions) and the size of a function's stack frame. So nm really isn't going to do you any good.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top