Why Virtual Memory always increase, nerver decrease

Question

I am maintaining a big system(very very big), and found the memory issue of it. That system is a single EXE with a lot of DLLs, written with C++, build with VS2008 and run on WinXP.

I use Process Explorer (sysinternals) to observe its memory usage. I find the gap between Virtual Memory (VM) and Private Bytes (PB) become larger and larger. At the beginning, VM is a little larger than PB, and then both VM and PB increase fast with the running. At some time, the PB decrease but the VM doesn't decrease. VM can become 300MB larger than PB.

I find VM always increase and PB increase and then decrease, at last VM is nearly 2G and the system crashes.

In my system, no direct calls to any virtual memory allocation functions, such as VirtualAlloc/VirtualAllocEx, just use new/delete/malloc/free/HeapCreate/HeapAlloc to use memory.

I use windbg to debug it, and use !heap -s to display the summary of process heap, and find this heap reserved more than 300MB memory, but just commit 15MB.

In this output, I also found some exceptions,

Virtual address fragmentation   -1340964299% (191 uncommited ranges)
ERROR Cannot read SubSegmentZones list at 79858461

Does anyone know the why VM always increase?
Thanks in advance.

Following is part output of !heap -s 00170000

!heap -s 00170000
Walking the heap 00170000 ...Verbose mode ON.
.......Virtual block: 26740000 - 26740000 (size 00000000)
Virtual block: 267f0000 - 267f0000 (size 00000000)

 0: Heap 00170000
   Flags          00000002 - HEAP_GROWABLE 
   Reserved memory in segments              327680 (k)
   Commited memory in segments              15548 (k)
   Virtual bytes (correction for large UCR) 884 (k)
   Free space                               5818 (k) (320 blocks)
   External fragmentation          37% (320 free blocks)
   Virtual address fragmentation   -1340964299% (191 uncommited ranges)
   Virtual blocks  2 - total 0 KBytes
   Lock contention 4
   Segments        10
   0 hash table for the free list
       Commits 0
       Decommitts 0

   Low fragmentation heap   15a3c9e0
ERROR Cannot read SubSegmentZones list at 79858461
       Metadata usage      3072 bytes
       Statistics:
           Segments created          0
           Segments deleted          0
           Segments reused           0

Answer 1

I am not sure if this will help, but I was having similar memory issues running memory intensive application under Windows XP. As I recall problem was memory fragmentation which was caused by lots of small stl container allocations/de-allocations. So if your application is doing lots of small allocations/de-allocations, I would suggest to reserve memory and re-use it.