Thanks to @Ascorbin I came to the answer! No need to reinvent the wheel. I used this library:
https://github.com/nostra13/Android-Universal-Image-Loader
If anyone has the same problem as me... That is... cached images getting reloaded from the interwebs.... Then use that library and this snippet to set your image:
DisplayImageOptions options = new DisplayImageOptions.Builder()
.showImageOnLoading(R.drawable.noimage) // resource or drawable
.showImageForEmptyUri(R.drawable.noimage) // resource or drawable
.resetViewBeforeLoading(true)
.delayBeforeLoading(0)
.cacheInMemory(true)
.cacheOnDisc(true)
.bitmapConfig(Bitmap.Config.ARGB_8888)
.build();
String url = d.getImageUrl();
ImageLoaderConfiguration config = new ImageLoaderConfiguration.Builder(v.getContext())
.defaultDisplayImageOptions(options)
.build();
ImageLoader imageLoader = ImageLoader.getInstance();
imageLoader.init(config);
imageLoader.displayImage("", holder.image);
imageLoader.displayImage(url, holder.image);