One thing you can try is using Bitmap.Recycle() after you finished with your bitmap to help the GC clean the object if you are not doing that already. Most likely this will help but not solve the problem.
The other thing is if you are using threads (AsyncTask would be a better option), are you managing the number of concurrent threads? For example, set a MAX number of downloads and don't start a new download until the number of concurrent downloads is less than MAX.
There is a nice way of downloading pics for a listview here, it may bring you some ideas. http://developer.android.com/training/displaying-bitmaps/process-bitmap.html