Iterator-like concept that generates data

Question

I have a (large) number of images files on the disk. I would like to iterate through them and do something for each image. I can not load all of them up front, I would like to load the image only when it's needed and dispose it just before I load the next one.

I implemented this with boost:iterator_facade and input iterator that loads the image in operator(). It works, but I encountered a number of difficulties which tell me that probably the iterators are intended to iterate over external sequence rather than to own the data.

I was wondering if there is some standard c++ idiom for this, sort of data owning iterator?

EDIT: the code I wrote is here

The "difficulties" are with

image_pair& dereference() const {...}

1. I would like to return image_pairthat is a part of the object, but since the dereference() is a const method, I cannot do that
2. Because of the same reason, I cannot load the images in dereference method (sice this changes the object), so I have to do it in increment()

Thanks

Answer 1

The iterator is expected to give a reference to a value, but that doesn't prevent it from managing that same value.

Consider the alternatives (1, 2):

1) So far I have found the functor interface the easiest to implement. A code using a functor might look like

auto images = Image::all(); while (auto image = images()) {image->doSomething();}

where Image::all returns std::function<boost::optional<Image>()> implemented by either a functor class or a lambda.

2) Rook's idea of using a stream looks really nice and idiomatic. Might look like

Images images; Image image; while (images >> image) {image.doSomething();}

where operator >> returns Images&, its operator bool() returning false when the stream is finished.

3) But I see the iterators used for that kind of thing all the time, like the recursive_directory_iterator. And the reason for this is the reuse of the generic algorithms, like std::find and std::copy. Imagine using std::copy to create a backup of your images:

std::copy (images.begin(), images.end(), backup.begin());

This is very succint, idiomatic and follows the principle of least astonishment.

I would say it's your choice, as a developer, whether to use a functor (aka generator), a stream, iterators or something else, but the iterators approach is usually the most friendly towards the users of your API.