Using boost range adaptors with a directory iterator range

Question

Edit: I added some solutions below my question, based on Jonathan's answer

I want to have a list of regular files with a certain name pattern in a given directory. I took one of the examples from boost.filesystem (boost 1.53) and modified it. Here is a working version of what I basically want:

#include <iostream>
#include <boost/filesystem.hpp>
#include <boost/foreach.hpp>

using namespace std;
using namespace boost::filesystem;

int main(int argc, char* argv[])
{
  path p ("."); // search directory

  try
  {
    if (exists(p))
    {
      if (is_directory(p))
      {
        cout << "logfiles in " << absolute(p) << '\n';
      // I want to replace this part:
        std::list<directory_entry> datFiles;
        for(directory_iterator it(p); it != directory_iterator(); ++it)
        {
          if (is_regular_file(*it) && (extension(*it) == ".dat"))
          {
            datFiles.push_back(*it);
          }
        }
      // part to be replaced ends here
        BOOST_FOREACH(const directory_entry& de, datFiles)
        {
          cout << de << '\n';
        }
      }
    }
  }

  catch (const filesystem_error& ex)
  {
    cout << ex.what() << '\n';
  }

  return 0;
}

This appends the entry of all files ending with .dat to the list of directory entries. I need more pattern matching later. I then tried to do the same thing with ranges, and failed. I now reduced the task to simply copying all directory entries which also doesn't compile:

    // replacement starts here
        std::list<directory_entry> datFiles;
        boost::iterator_range<directory_iterator> rng(directory_iterator(p), directory_iterator());should be
        boost::copy(rng, datFiles.begin());
    // replacement ends here

I wrote it this way because (tut3.cpp from boost.filesystem)

directory_iterator::value_type is directory_entry

So that copy can iterate over the range, get entries and store them in my list. However, I'm getting lots of error messages from gcc:

In file included from boost_1_53_0/boost/type_traits/decay.hpp:18:0,
                 from boost_1_53_0/boost/filesystem/path_traits.hpp:22,
                 from boost_1_53_0/boost/filesystem/path.hpp:25,
                 from boost_1_53_0/boost/filesystem.hpp:16,
                 from ~/20130519_searchDatFiles/main.cpp:3:
boost_1_53_0/boost/mpl/eval_if.hpp: In instantiation of ‘boost::mpl::eval_if_c<false, boost::range_const_iterator<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())>, boost::range_mutable_iterator<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())> >’:
boost_1_53_0/boost/range/iterator.hpp:63:63:   instantiated from ‘boost::range_iterator<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())>’
boost_1_53_0/boost/range/concepts.hpp:256:72:   instantiated from ‘boost::SinglePassRangeConcept<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())>’
boost_1_53_0/boost/concept/detail/has_constraints.hpp:42:5:   instantiated from ‘const bool boost::concepts::not_satisfied<boost::SinglePassRangeConcept<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())> >::value’
boost_1_53_0/boost/concept/detail/has_constraints.hpp:45:31:   instantiated from ‘boost::concepts::not_satisfied<boost::SinglePassRangeConcept<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())> >’
boost_1_53_0/boost/mpl/if.hpp:67:11:   instantiated from ‘boost::mpl::if_<boost::concepts::not_satisfied<boost::SinglePassRangeConcept<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())> >, boost::concepts::constraint<boost::SinglePassRangeConcept<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())> >, boost::concepts::requirement<boost::concepts::failed************ boost::SinglePassRangeConcept<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())>::************> >’
boost_1_53_0/boost/concept/detail/general.hpp:50:8:   instantiated from ‘boost::concepts::requirement_<void (*)(boost::SinglePassRangeConcept<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())>)>’
boost_1_53_0/boost/range/algorithm/copy.hpp:33:1:   instantiated from ‘OutputIterator boost::range::copy(const SinglePassRange&, OutputIterator) [with SinglePassRange = boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)()), OutputIterator = std::_List_iterator<boost::filesystem::directory_entry>]’
~/20130519_searchDatFiles/main.cpp:45:42:   instantiated from here
boost_1_53_0/boost/mpl/eval_if.hpp:60:31: error: no type named ‘type’ in ‘boost::mpl::eval_if_c<false, boost::range_const_iterator<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())>, boost::range_mutable_iterator<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())> >::f_ {aka struct boost::range_mutable_iterator<boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)())>}’
In file included from boost_1_53_0/boost/range/algorithm.hpp:53:0,
                 from ~/20130519_searchDatFiles/main.cpp:7:
boost_1_53_0/boost/range/algorithm/copy.hpp: In function ‘OutputIterator boost::range::copy(const SinglePassRange&, OutputIterator) [with SinglePassRange = boost::iterator_range<boost::filesystem::directory_iterator>(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)()), OutputIterator = std::_List_iterator<boost::filesystem::directory_entry>]’:
~/20130519_searchDatFiles/main.cpp:45:42:   instantiated from here
boost_1_53_0/boost/range/algorithm/copy.hpp:34:59: error: no matching function for call to ‘begin(boost::iterator_range<boost::filesystem::directory_iterator> (&)(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)()))’
boost_1_53_0/boost/range/algorithm/copy.hpp:34:59: note: candidates are:
boost_1_53_0/boost/range/begin.hpp:101:55: note: template<class T> typename boost::range_iterator::type boost::range_adl_barrier::begin(T&)
boost_1_53_0/boost/range/begin.hpp:112:61: note: template<class T> typename boost::range_iterator<const T>::type boost::range_adl_barrier::begin(const T&)
boost_1_53_0/boost/range/algorithm/copy.hpp:34:59: error: no matching function for call to ‘end(boost::iterator_range<boost::filesystem::directory_iterator> (&)(boost::filesystem::directory_iterator, boost::filesystem::directory_iterator (*)()))’
boost_1_53_0/boost/range/algorithm/copy.hpp:34:59: note: candidates are:
boost_1_53_0/boost/range/end.hpp:95:55: note: template<class T> typename boost::range_iterator::type boost::range_adl_barrier::end(T&)
boost_1_53_0/boost/range/end.hpp:106:61: note: template<class T> typename boost::range_iterator<const T>::type boost::range_adl_barrier::end(const T&)
boost_1_53_0/boost/range/algorithm/copy.hpp:35:1: warning: control reaches end of non-void function [-Wreturn-type]

I wasn't really able to filter out the relevant lines, but I think that it is possible to filter files using range adaptors (I don't just want to copy the range, after all). My impression is that somewhere along the way datFiles.begin() doesn't satisfy a concept needed by the copy algorithm.

---

Some of my solutions

Here are some of the solutions (including filtering) I came up with, based on Jonathan's answer.

Using a global predicate function bool isDatFile(const path& p):

std::list<directory_entry> datFiles;
boost::copy(rng | boost::adaptors::filtered(isDatFile), std::back_inserter(datFiles));

I preferred a local function, though, so I tried that. This one is just checking if the directory entry is a regular file, the static cast was needed because is_regular_file is overloaded:

std::list<directory_entry> datFiles;
boost::copy(rng
            | boost::adaptors::filtered(static_cast<bool (*)(const path&)>(&is_regular_file)),
            std::back_inserter(datFiles));

Checking for the right extension in the range adaptor as well was a bit hard, so I added a local function:

std::list<directory_entry> datFiles;
bool BOOST_LOCAL_FUNCTION(const path& p)
{
  return (is_regular_file(p) && (extension(p) == string(".dat")));
} BOOST_LOCAL_FUNCTION_NAME(isDatFile_l)
boost::copy(rng
            | boost::adaptors::filtered(isDatFile_l),
            std::back_inserter(datFiles));

And finally the "one-liner" using bind:

std::list<directory_entry> datFiles;
boost::copy(rng
            | boost::adaptors::filtered(static_cast<bool (*)(const path&)>(&is_regular_file))
            | boost::adaptors::filtered(boost::bind(std::equal_to<path>(), boost::bind(extension, _1), path(".dat"))),
            std::back_inserter(datFiles));

Answer 1

This line is hitting is the Most Vexing Parse:

boost::iterator_range<directory_iterator> rng(directory_iterator(p), directory_iterator());

That declares a function not an object.

You can change it to:

boost::iterator_range<directory_iterator> rng = boost::iterator_range<directory_iterator>(directory_iterator(p), directory_iterator());

Or in C++11:

auto rng = boost::iterator_range<directory_iterator>(directory_iterator(p), directory_iterator());

But it's even better to use the factory function for it:

boost::iterator_range<directory_iterator> rng = boost::make_iterator_range(directory_iterator(p), directory_iterator());

By using make_iterator_range you don't need to name the type of iterator, it will be deduced from the argument types.

The other problem is that you're writing to an empty list, which is undefined behaviour, you should use an insert iterator to call push_back() like in your working code. Putting it all together:

std::list<directory_entry> datFiles;
boost::copy( boost::make_iterator_range(directory_iterator(p), directory_iterator()),
             std::back_inserter(datFiles) );

In C++11 it's more concise:

std::list<directory_entry> datFiles;
boost::copy( boost::make_iterator_range(directory_iterator(p), {}),
             std::back_inserter(datFiles) );

Now you could add the filter pretty easily:

std::list<directory_entry> datFiles;
boost::copy( boost::make_iterator_range(directory_iterator(p), {}) | some_dir_filter,
             std::back_inserter(datFiles) );