SEH StackOverflow exception - is it real not possible to catch?

Question

I have read many articles about SEH exceptions in StackOverflow and CodeProject.net.

After I implemented SEH exceptions handling in my C++ program, I was affected by stack overflow exception, which hadn't been caught by my software.

After next part of research I understand, that it's impossible to detect such exception programmatically, because we don't have free stack address space to use, so program memory is corrupted.

I would like to ask you about your experience in handling stack overflow exception. It looks like a challenge and I'm really interested if it's not possible in unmanaged code programming languages?

Below I present a part of my sample program (C++), which reproduces stack overflow exception. It works perfectly for any SEH exception, but not stack overflow:

LONG WINAPI SehHandler(PEXCEPTION_POINTERS pExceptionPtrs)
{ 
    cerr << "Handled SEH exception!\n";
    cerr << "ContextRecord: " << pExceptionPtrs->ContextRecord << endl;
    cerr << "ExceptionRecord: " << pExceptionPtrs->ExceptionRecord << endl;

    // Write minidump file
    CreateMiniDump(pExceptionPtrs);

    // Terminate process
    TerminateProcess(GetCurrentProcess(), 1); 

    return EXCEPTION_EXECUTE_HANDLER;
}

int fib(unsigned int n) {
    if(n == 0) return 0;
    if(n == 1) return 1;
    return fib(n-1)+fib(n-2);
}

int main(){
    SetUnhandledExceptionFilter(SehHandler); 
    cout << fib(1000000);
    return 0;
}

Answer 1

Yes, you can get a minidump out of a SO crash, but never the way you are doing it now. Your SehHandler() function runs on the thread that triggered the exception. And it is in a perilous state, you have about ~7080 bytes of emergency stack space left to do what you need to do. If you consume that then the program will fail with an uncatchable access violation exception.

You cannot call MiniDumpWriteDump() and hope to survive it, that function requires more stack than you have available. So it is a hard kaboom without a minidump.

You need another thread to make that call. That could be, for example, a thread that you create at initialization and block with a WaitForMultipleObjects() call. Your SehHandler() can call SetEvent() to wake it up. After writing the PEXCEPTION_POINTERS value to a global variable. And block indefinitely to allow the thread to create the minidump and abort the process.

Fwiw, by far the best place for that thread is in another process. That also allows you to deal with the really nasty ones that completely corrupt the process state. A "guard" process that you start at initialization. With a named event to signal it and, say, a memory-mapped file to pass the PEXCEPTION_POINTERS. Don't start it in SehHandler(), the process heap is no longer reliable so CreateProcess() cannot work anymore, you have to do it early.

Answer 2

The answer from Hans Passant indicates that there's 7080 bytes of emergency stack. I don't know where that info comes from, and he hasn't answered @nop above, and my findings indicate that that info is incorrect. However this site won't allow me to comment above for some reason, so I'll just leave this here...

There's a function that can be used to query and set how much emergency stack is left to the stack handler: SetThreadStackGuarantee(). Note that since Windows 10 definitely (but I think it's since Windows 7 as well), in most cases this value will be 0. So there's no way to do anything complicated in the handler, by default. You may be able to signal another thread or an external process, as Hans has suggested, but that's all.

However, if you don't want to implement such a complicated solution, and can spare some slack space on the stack, it is easiest to use SetThreadStackGuarantee() to set this to a value high enough that you can continue handling the stack overflow exception just like any other. Note that you need to call this function on each thread that needs this feature, and call it before the stack overflow occurs, so preferably at thread initialization.