debug stack overflow in windows?

Question

So I'm trying to debug this strange problem where a process ends without calling some destructors...

In the VS (2005) debugger, I hit 'Break all' and look around in the call stacks of the threads of the misteriously disappearing process, when I see this:

smells like SO http://img6.imageshack.us/img6/7628/95434880.jpg

This definitely looks like a SO in the making, which would explain why the process runs to its happy place without packing its suitcase first.

The problem is, the VS debugger's call stack only shows what you can see in the image.

So my question is: how can I find where the infinite recursion call starts?

I read somewhere that in Linux you can attach a callback to the SIGSEGV handler and get more info on what's going on.

Is there anything similar on Windows?

Answer 1

To control what Windows does in case of an access violation (SIGSEGV-equivalent), call SetErrorMode (pass it parameter 0 to force a popup in case of errors, allowing you to attach to it with a debugger.)

However, based on the stack trace you have already obtained, attaching with a debugger on fault may yield no additional information. Either your stack has been corrupted, or the depth of recursion has exceeded the maximum number of frames displayable by VS. In the latter case, you may want to decrease the default stack size of the process (use the /F switch or equivalent option in the Project properties) in order to make the problem manifest itself sooner, and make sure that VS will display all frames. You may, alternatively, want to stick a breakpoint in std::basic_filebuf<>::flush() and walk through it until the destruction phase (or disable it until just prior to the destruction phase.)

Answer 2

Well, you know what thread the problem is on - it might be a simple matter of tracing through it from inception to see where it goes off into the weeds.

Another option is to use one of the debuggers in the Debugging Tools for Windows package - they may be able to show more than the VS debugger (maybe), even if they are generally more complex and difficult to use (actually maybe because of that).

Answer 3

That does look at first glance like an infinite recursion, you could try putting a breakpoint at the line before the one that terminates the process. Does it get there ok? If it does, you've got two fairly easy ways to go.

Either you just step forward and see which destructors get called and when it gets caught up. Or you could put a printf/OutputDebugString in every relevant objects destructor (ONly ones which are globals should need this). If the message is the first thing the destructor does, then the last message you see is from the destructor which hangs things up.

On the other hand, if it doesn't get to that breakpoint I originally mentioned, then can do something similar, but it will be more annoying since the program is still "doing stuff".

Answer 4

I wouldn't rule out there being such a handler in Windows, but I've never heard of it.

I think the traceback that you're showing may be bogus. If you broke into the process after some kind of corruption had already occurred, then the traceback isn't necessarily valid. However, if you're lucky the bottom of the stack trace still has some clues about what's going on.

Try putting Sleep() calls into selected functions in your source that might be involved in the recursion. That should give you a better chance of breaking into the process before the stack has completely overflowed.

Answer 5

I agree with Dan Breslau. Your stack is bogus. may be simply because you don't have the right symbols, though. If a program simply disappears without the WER handling kicking in, it's usually an out of memory condition. Have you gone investigated that possibility ?