WebSphere® Message Broker - DataFlowEngine Process - Failed to allocate memory

Question

The WebSphere Message Broker DataFlowEngine Process abnormally ends with an error Failed to allocate memory

This happens when the incoming XML message is roughly 12 Mb.

It was also noticed that a similar incoming XML message but 10 Mb in size did not lead to this problem.

I have attached 3 Examples of the .abend log file generated at the time of processing this message. They are at

1. http://dpaste.com/1356207/
2. http://dpaste.com/1356208/
3. http://dpaste.com/1356209/

We have so far tried setting the jvmMaxHeapSize to 1 GB but that did not help.....the logs generated in /var/adm/log after setting this value are at http://dpaste.com/1356210/

The size of the DataFlowEngine process increases quite a lot before it abnormally ended. I was able to see it reaching to a size of 1.2 GB via NMON utility.

We also tried setting the variable MQSI_FREE_MASTER_PARSERS=1 based on some link I found on Google but it did not help either.

What else could we do to get to the root of this issue?

Answer 1

The DFE process is trying to allocate more memory in order to create a syntax element for the target of a tree copy in an ESQL compute node.

The most likely explanation is that you are copying an enormous tree and the DFE needs more memory than it is allowed to allocate to process the message.

It is worth bearing in mind that the size on an XML input file can be many times smaller than the size required just to store one copy of the syntax tree representing the document as in addition to the bitstream stored in each element the broker has to store pointers to parent and child elements, type data and structure data.

Further this kind of data needs to be stored for every tree copy that you do meaning that even small XML input documents, if not handled properly can require a huge amount of memory to process.

Judging from your abends I would say your flow is failing at the end of a section that looks like this:

MQInput Node -> Compute Node -> Try/CatchNode -> Compute Node -> Compute Node -> Compute Node

I would review all of your compute nodes starting from the right most for unrequired tree copies. It is also worth reviewing the following Infocenter topic which gives hints on how to use reference variables to reduce the amount of storage required when processing repeating structures in ESQL:

http://publib.boulder.ibm.com/infocenter/wmbhelp/v8r0m0/topic/com.ibm.etools.mft.doc/ac67176_.htm http://publib.boulder.ibm.com/infocenter/wmbhelp/v8r0m0/topic/com.ibm.etools.mft.doc/bc23802_.htm