How to release DataSnap memory once connections are closed?

Question

I have a TCP/IP DataSnap server running as a service [Session based LifeCycle] that continuously chews up memory and never comes back to the starting memory size even when there are no connections to it.

In order to eliminate My code as the culprit, I have modeled up a basic TCP/IP DataSnap server running as VCL [Session based LifeCycle] that serves a Server Method class [TDSServerModule] which only contains basic mathematical functions using native data types [no objects to create or free].

When I connect to said DataSnap server with a very thin client I get the same results. The Memory Usage continuously grows with each connection and sporadically grows when executing the server side methods from the client. Once the connections are closed the DataSnap Server never reduces its Memory Usage [even when left running without connections for 8 hrs].

Any suggestions as to why this occurs or more importantly how to curtail it?

I am using RAD Studio XE2 Update 4 HotFix 1.

Answer 1

Let me quote a "must read" article about DataSnap. This is about XE3 but i hope the code here would work for XE2 as well.

Memory consumption

One of the issues that I observed was related to memory consumption. Why Datasnap server consumes so much memory if the method called does absolutely nothing?

Maybe I don’t know how to explain exactly but i will try. Basically the DataSnap creates a session for each HTTP connection that it receives. This session will be destroyed after 20 minutes, in other words, on the first 20 minutes of the test the memory consumption will only go up, after that it has the tendency of stabilize itself. I really have no idea why Datasnap does this. In a REST application I don’t see much sense in these sessions with a default configuration. Of course, sessions can be helpful, but i can’t understand why it’s a default configuration. Indeed, DataSnap doesn’t have a configuration for that. It appears like you just have to use this session control, without being able to choose otherwise (There is no documentation). The MORMot framework has a session control too but it’s configurable and doesn’t consumes so much memory.

Anyway, there is a way around this problem. Daniele Teti wrote an article on his blog, take a look. The solution that I will show here was placed by him on his blog. Thanks Daniele.

uses System.StrUtils, DataSnap.DSSession, Data.DBXPlatform;

function TServerMethods1.HelloWorld: String;
 begin

 Result := 'Hello World';
 GetInvocationMetaData.CloseSession := True;
end;

After running this method the session will close and memory consumption will be lower. Of course still exists an overhead for creating and destroying this session.

So it seems the best course for you is ending every server method with explicit memory cleanup, if that was possible in XE2. Then you'd better read thosee articles again and prepare for future scalability challenges.

• http://www.danieleteti.it/2012/12/15/datasnap-concurrency-problems-and-update1/
• http://robertocschneiders.wordpress.com/2013/01/09/datasnap-analysis-based-on-speed-stability-tests-part-2/

Answer 2

I added the below method and called it from the "TWebModule1::WebModuleBeforeDispatch" event. It eliminated the memory consumption and actually allow the idle REST service to return to a state of no-session memory. DataSnap definitely needs to work on this issue.

// ---------------------------------------------------------------------------
/// <summary> Memory Restoration. DataSnap opens a session for each call
///         even when the service is set for invocation.
///         Sessions are building up consuming memory and seem not to be freed.
///         See: https://stackoverflow.com/questions/17748300/how-to-release-datasnap-memory-once-connections-are-closed
/// </summary>
/// <remarks> Iterates session in the session manager and closes then terminates
///     any session that has been idle for over 10 seconds.
/// </remarks>
/// <returns> void
/// </returns>
// ---------------------------------------------------------------------------
void TWebModule1::CloseIdleSessions()
{
TDSSessionManager* sessMgr = TDSSessionManager::Instance;
int sessCount = sessMgr->GetSessionCount();
WriteLogEntry(LogEntryTypeDebug, "TWebModule1::CloseIdleSessions", "Session Count: " + IntToStr(sessCount));
TStringList* sessKeys = new TStringList;
sessMgr->GetOpenSessionKeys(sessKeys);
WriteLogEntry(LogEntryTypeDebug, "TWebModule1::CloseIdleSessions", "Session Keys Count: " + IntToStr(sessKeys->Count));
TDSSession* sess = NULL;
for(int index = 0; index < sessKeys->Count; index++)
{
    String sessKey = sessKeys->Strings[index];
    sess = sessMgr->Session[sessKey];
    unsigned elapsed = (int)sess->ElapsedSinceLastActvity();
    if(elapsed > 10000)
    {
        WriteLogEntry(LogEntryTypeDebug, "TWebModule1::CloseIdleSessions", "CloseSession TerminateSession Key: " + sessKey);
        sessMgr->CloseSession(sessKey);
        sessMgr->TerminateSession(sessKey);
    }
    sess = NULL;
}
delete sessKeys;
sessMgr = NULL;
}

Answer 3

You should check the Lifecycle property on TDSServerclass component on your servercontainer. It provides a way to determine the way the session is handled. It defaults to session. Setting it to invokation will free the session after each call (invokation). This will of course mean you have no state. This would be oke in a typical REST server though.

if you still have memory consumption growing. put the following line in your dpr unit. ReportMemoryLeaksOnShutdown := True;
your application will then show you the memory leaks it has on closing of your datasnap server.