How can I schedule some code to run after all '_atexit()' functions are completed

Question

I'm writing a memory tracking system and the only problem I've actually run into is that when the application exits, any static/global classes that didn't allocate in their constructor, but are deallocating in their deconstructor are deallocating after my memory tracking stuff has reported the allocated data as a leak.

As far as I can tell, the only way for me to properly solve this would be to either force the placement of the memory tracker's _atexit callback at the head of the stack (so that it is called last) or have it execute after the entire _atexit stack has been unwound. Is it actually possible to implement either of these solutions, or is there another solution that I have overlooked.

Edit: I'm working on/developing for Windows XP and compiling with VS2005.

Answer 1

I've finally figured out how to do this under Windows/Visual Studio. Looking through the crt startup function again (specifically where it calls the initializers for globals), I noticed that it was simply running "function pointers" that were contained between certain segments. So with just a little bit of knowledge on how the linker works, I came up with this:

#include <iostream>
using std::cout;
using std::endl;

// Typedef for the function pointer
typedef void (*_PVFV)(void);

// Our various functions/classes that are going to log the application startup/exit
struct TestClass
{
    int m_instanceID;

    TestClass(int instanceID) : m_instanceID(instanceID) { cout << "  Creating TestClass: " << m_instanceID << endl; }
    ~TestClass() {cout << "  Destroying TestClass: " << m_instanceID << endl; }
};
static int InitInt(const char *ptr) { cout << "  Initializing Variable: " << ptr << endl; return 42; }
static void LastOnExitFunc() { puts("Called " __FUNCTION__ "();"); }
static void CInit() { puts("Called " __FUNCTION__ "();"); atexit(&LastOnExitFunc); }
static void CppInit() { puts("Called " __FUNCTION__ "();"); }

// our variables to be intialized
extern "C" { static int testCVar1 = InitInt("testCVar1"); }
static TestClass testClassInstance1(1);
static int testCppVar1 = InitInt("testCppVar1");

// Define where our segment names
#define SEGMENT_C_INIT      ".CRT$XIM"
#define SEGMENT_CPP_INIT    ".CRT$XCM"

// Build our various function tables and insert them into the correct segments.
#pragma data_seg(SEGMENT_C_INIT)
#pragma data_seg(SEGMENT_CPP_INIT)
#pragma data_seg() // Switch back to the default segment

// Call create our call function pointer arrays and place them in the segments created above
#define SEG_ALLOCATE(SEGMENT)   __declspec(allocate(SEGMENT))
SEG_ALLOCATE(SEGMENT_C_INIT) _PVFV c_init_funcs[] = { &CInit };
SEG_ALLOCATE(SEGMENT_CPP_INIT) _PVFV cpp_init_funcs[] = { &CppInit };


// Some more variables just to show that declaration order isn't affecting anything
extern "C" { static int testCVar2 = InitInt("testCVar2"); }
static TestClass testClassInstance2(2);
static int testCppVar2 = InitInt("testCppVar2");


// Main function which prints itself just so we can see where the app actually enters
void main()
{
    cout << "    Entered Main()!" << endl;
}

which outputs:

Called CInit();
Called CppInit();
  Initializing Variable: testCVar1
  Creating TestClass: 1
  Initializing Variable: testCppVar1
  Initializing Variable: testCVar2
  Creating TestClass: 2
  Initializing Variable: testCppVar2
    Entered Main()!
  Destroying TestClass: 2
  Destroying TestClass: 1
Called LastOnExitFunc();

This works due to the way MS have written their runtime library. Basically, they've setup the following variables in the data segments:

(although this info is copyright I believe this is fair use as it doesn't devalue the original and IS only here for reference)

extern _CRTALLOC(".CRT$XIA") _PIFV __xi_a[];
extern _CRTALLOC(".CRT$XIZ") _PIFV __xi_z[];    /* C initializers */
extern _CRTALLOC(".CRT$XCA") _PVFV __xc_a[];
extern _CRTALLOC(".CRT$XCZ") _PVFV __xc_z[];    /* C++ initializers */
extern _CRTALLOC(".CRT$XPA") _PVFV __xp_a[];
extern _CRTALLOC(".CRT$XPZ") _PVFV __xp_z[];    /* C pre-terminators */
extern _CRTALLOC(".CRT$XTA") _PVFV __xt_a[];
extern _CRTALLOC(".CRT$XTZ") _PVFV __xt_z[];    /* C terminators */

On initialization, the program simply iterates from '__xN_a' to '__xN_z' (where N is {i,c,p,t}) and calls any non null pointers it finds. If we just insert our own segment in between the segments '.CRT$XnA' and '.CRT$XnZ' (where, once again n is {I,C,P,T}), it will be called along with everything else that normally gets called.

The linker simply joins up the segments in alphabetical order. This makes it extremely simple to select when our functions should be called. If you have a look in defsects.inc (found under $(VS_DIR)\VC\crt\src\) you can see that MS have placed all the "user" initialization functions (that is, the ones that initialize globals in your code) in segments ending with 'U'. This means that we just need to place our initializers in a segment earlier than 'U' and they will be called before any other initializers.

You must be really careful not to use any functionality that isn't initialized until after your selected placement of the function pointers (frankly, I'd recommend you just use .CRT$XCT that way its only your code that hasn't been initialized. I'm not sure what will happen if you've linked with standard 'C' code, you may have to place it in the .CRT$XIT block in that case).

One thing I did discover was that the "pre-terminators" and "terminators" aren't actually stored in the executable if you link against the DLL versions of the runtime library. Due to this, you can't really use them as a general solution. Instead, the way I made it run my specific function as the last "user" function was to simply call atexit() within the 'C initializers', this way, no other function could have been added to the stack (which will be called in the reverse order to which functions are added and is how global/static deconstructors are all called).

Just one final (obvious) note, this is written with Microsoft's runtime library in mind. It may work similar on other platforms/compilers (hopefully you'll be able to get away with just changing the segment names to whatever they use, IF they use the same scheme) but don't count on it.

Answer 2

atexit is processed by the C/C++ runtime (CRT). It runs after main() has already returned. Probably the best way to do this is to replace the standard CRT with your own.

On Windows tlibc is probably a great place to start: http://www.codeproject.com/KB/library/tlibc.aspx

Look at the code sample for mainCRTStartup and just run your code after the call to _doexit(); but before ExitProcess.

Alternatively, you could just get notified when ExitProcess gets called. When ExitProcess gets called the following occurs (according to http://msdn.microsoft.com/en-us/library/ms682658%28VS.85%29.aspx):

1. All of the threads in the process, except the calling thread, terminate their execution without receiving a DLL_THREAD_DETACH notification.
2. The states of all of the threads terminated in step 1 become signaled.
3. The entry-point functions of all loaded dynamic-link libraries (DLLs) are called with DLL_PROCESS_DETACH.
4. After all attached DLLs have executed any process termination code, the ExitProcess function terminates the current process, including the calling thread.
5. The state of the calling thread becomes signaled.
6. All of the object handles opened by the process are closed.
7. The termination status of the process changes from STILL_ACTIVE to the exit value of the process.
8. The state of the process object becomes signaled, satisfying any threads that had been waiting for the process to terminate.

So, one method would be to create a DLL and have that DLL attach to the process. It will get notified when the process exits, which should be after atexit has been processed.

Obviously, this is all rather hackish, proceed carefully.

Answer 3

This is dependent on the development platform. For example, Borland C++ has a #pragma which could be used for exactly this. (From Borland C++ 5.0, c. 1995)

#pragma startup function-name [priority]
#pragma exit    function-name [priority]

These two pragmas allow the program to specify function(s) that should be called either upon program startup (before the main function is called), or program exit (just before the program terminates through _exit). The specified function-name must be a previously declared function as:

void function-name(void);

The optional priority should be in the range 64 to 255, with highest priority at 0; default is 100. Functions with higher priorities are called first at startup and last at exit. Priorities from 0 to 63 are used by the C libraries, and should not be used by the user.

Perhaps your C compiler has a similar facility?

Answer 4

I've read multiple times you can't guarantee the construction order of global variables (cite). I'd think it is pretty safe to infer from this that destructor execution order is also not guaranteed.

Therefore if your memory tracking object is global, you will almost certainly be unable any guarantees that your memory tracker object will get destructed last (or constructed first). If it's not destructed last, and other allocations are outstanding, then yes it will notice the leaks you mention.

Also, what platform is this _atexit function defined for?

Answer 5

Having the memory tracker's cleanup executed last is the best solution. The easiest way I've found to do that is to explicitly control all the relevant global variables' initialization order. (Some libraries hide their global state in fancy classes or otherwise, thinking they're following a pattern, but all they do is prevent this kind of flexibility.)

Example main.cpp:

#include "global_init.inc"
int main() {
  // do very little work; all initialization, main-specific stuff
  // then call your application's mainloop
}

Where the global-initialization file includes object definitions and #includes similar non-header files. Order the objects in this file in the order you want them constructed, and they'll be destructed in the reverse order. 18.3/8 in C++03 guarantees that destruction order mirrors construction: "Non-local objects with static storage duration are destroyed in the reverse order of the completion of their constructor." (That section is talking about exit(), but a return from main is the same, see 3.6.1/5.)

As a bonus, you're guaranteed that all globals (in that file) are initialized before entering main. (Something not guaranteed in the standard, but allowed if implementations choose.)

Answer 6

I've had this exact problem, also writing a memory tracker.

A few things:

Along with destruction, you also need to handle construction. Be prepared for malloc/new to be called BEFORE your memory tracker is constructed (assuming it is written as a class). So you need your class to know whether it has been constructed or destructed yet!

class MemTracker
{
    enum State
    {
      unconstructed = 0, // must be 0 !!!
      constructed,
      destructed
    };
    State state;

    MemTracker()
    {
       if (state == unconstructed)
       {
          // construct...
          state = constructed;
       }
    }
};

static MemTracker memTracker;  // all statics are zero-initted by linker

On every allocation that calls into your tracker, construct it!

MemTracker::malloc(...)
{
    // force call to constructor, which does nothing after first time
    new (this) MemTracker();
    ...
}

Strange, but true. Anyhow, onto destruction:

    ~MemTracker()
    {
        OutputLeaks(file);
        state = destructed;
    }

So, on destruction, output your results. Yet we know that there will be more calls. What to do? Well,...

   MemTracker::free(void * ptr)
   {
      do_tracking(ptr);

      if (state == destructed)
      {
          // we must getting called late
          // so re-output
          // Note that this might happen a lot...
          OutputLeaks(file); // again!
       }
   }

And lastly:

• be careful with threading
• be careful not to call malloc/free/new/delete inside your tracker, or be able to detect the recursion, etc :-)

EDIT:

• and I forgot, if you put your tracker in a DLL, you will probably need to LoadLibrary() (or dlopen, etc) yourself to up your reference count, so that you don't get removed from memory prematurely. Because although your class can still be called after destruction, it can't if the code has been unloaded.