Java optimization to prevent heapspace out of memory

Question

Ok, I have a problem in a particular situation that my program get the out of memory error from heap space.

Let's assume we have two ArrayList, the first one contains many T objects, the second one contains W object that are created from the T objects of first List.

And we cycle through it in this way (after the cycle the list :

public void funct(ArrayList<T> list)
{
  ArrayList<W> list2 = new ArrayList<W>();
  for (int i = 0 ; i < list.size() ; i++)
  {
     W temp = new W();
     temp.set(list.get(i));
     temp.saveToDB();
     list2.add(temp);          
  }

  // more code! from this point on the `list` is useless
}

My code is pretty similar to this one, but when list contains tons of objects I often get the heap space out of memory (during the for cycle), I'd like to solve this problem.

I do not know very well how the GC works in java, but surely in the previous example there are a lot of possible optimization. Since the list is not used anymore after the for cycle I thought as first optimization to change from for loop to do loop and empty the list as we cycle through it:

public void funct(ArrayList<T> list)
{
  ArrayList<W> list2 = new ArrayList<W>();
  while (list.size() > 0)
  {
     W temp = new W();
     temp.set(list.remove(0));
     temp.saveToDB();
     list2.add(temp);
  }

  // more code! from this point on the `list` is useless
}

Is this modification useful? How can I do a better optimization to the above code? and how can I prevent heap space out of memory error? (increasing the XMX and XMS value is not a possibility).

Answer 1

You can try to set the -XX:MaxNewSize=40% of you Xmx AND -XX:NewSize=40% of you Xmx This params will speedup the GC calls, because your creation rate is high.

For more help : check here

Answer 2

It really depends on many things. How big are the W and T objects? One optimization you could surely do is ArrayList list2 = new ArrayList(list.size()); This way your listarray does not need to adjust its size many times. That will not do much difference tough. The real problem is probably the size and number of your W and T objects. Have you thought of using different data structures to manage a smaller portion of objects at time?

Answer 3

If you did some memory profiling, you would have discovered that the largest source of heap exhaustion are the W instances, which you retain by adding them to list2. The ArrayList itself adds a very small overhead per object contained (just 4 bytes if properly pre-sized, worst case 8 bytes), so even if you retain list, this cannot matter much.

You will not be able to lessen the heap pressure without changing your approach towards the non-retention of each and every W instance you have created in your loop.

Answer 4

You continue to reference all the items from the original list :

temp.set(list.get(i)) // you probably store somewhere the passed reference

If the T object has a big size and you don't need all of its fields, try to use a projection of it.

temp.set( extractWhatINeed( list.get(i) ) )

This will involve creating a new class with fewer fields than T (the return type of the extract method).

Now, when you don't reference the original items, they are eligible for GC (when the list itself will not be referenced anymore).