How do you iterate through every file/directory recursively in standard C++?

Question

How do you iterate through every file/directory recursively in standard C++?

Answer 1

In standard C++, technically there is no way to do this since standard C++ has no conception of directories. If you want to expand your net a little bit, you might like to look at using Boost.FileSystem. This has been accepted for inclusion in TR2, so this gives you the best chance of keeping your implementation as close as possible to the standard.

An example, taken straight from the website:

bool find_file( const path & dir_path,         // in this directory,
                const std::string & file_name, // search for this name,
                path & path_found )            // placing path here if found
{
  if ( !exists( dir_path ) ) return false;
  directory_iterator end_itr; // default construction yields past-the-end
  for ( directory_iterator itr( dir_path );
        itr != end_itr;
        ++itr )
  {
    if ( is_directory(itr->status()) )
    {
      if ( find_file( itr->path(), file_name, path_found ) ) return true;
    }
    else if ( itr->leaf() == file_name ) // see below
    {
      path_found = itr->path();
      return true;
    }
  }
  return false;
}

Answer 2

If using the Win32 API you can use the FindFirstFile and FindNextFile functions.

http://msdn.microsoft.com/en-us/library/aa365200(VS.85).aspx

For recursive traversal of directories you must inspect each WIN32_FIND_DATA.dwFileAttributes to check if the FILE_ATTRIBUTE_DIRECTORY bit is set. If the bit is set then you can recursively call the function with that directory. Alternatively you can use a stack for providing the same effect of a recursive call but avoiding stack overflow for very long path trees.

#include <windows.h>
#include <string>
#include <vector>
#include <stack>
#include <iostream>

using namespace std;

bool ListFiles(wstring path, wstring mask, vector<wstring>& files) {
    HANDLE hFind = INVALID_HANDLE_VALUE;
    WIN32_FIND_DATA ffd;
    wstring spec;
    stack<wstring> directories;

    directories.push(path);
    files.clear();

    while (!directories.empty()) {
        path = directories.top();
        spec = path + L"\\" + mask;
        directories.pop();

        hFind = FindFirstFile(spec.c_str(), &ffd);
        if (hFind == INVALID_HANDLE_VALUE)  {
            return false;
        } 

        do {
            if (wcscmp(ffd.cFileName, L".") != 0 && 
                wcscmp(ffd.cFileName, L"..") != 0) {
                if (ffd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
                    directories.push(path + L"\\" + ffd.cFileName);
                }
                else {
                    files.push_back(path + L"\\" + ffd.cFileName);
                }
            }
        } while (FindNextFile(hFind, &ffd) != 0);

        if (GetLastError() != ERROR_NO_MORE_FILES) {
            FindClose(hFind);
            return false;
        }

        FindClose(hFind);
        hFind = INVALID_HANDLE_VALUE;
    }

    return true;
}

int main(int argc, char* argv[])
{
    vector<wstring> files;

    if (ListFiles(L"F:\\cvsrepos", L"*", files)) {
        for (vector<wstring>::iterator it = files.begin(); 
             it != files.end(); 
             ++it) {
            wcout << it->c_str() << endl;
        }
    }
    return 0;
}

Answer 3

You can make it even simpler with the new C++11 range based for and Boost:

#include <boost/filesystem.hpp>

using namespace boost::filesystem;    
struct recursive_directory_range
{
    typedef recursive_directory_iterator iterator;
    recursive_directory_range(path p) : p_(p) {}

    iterator begin() { return recursive_directory_iterator(p_); }
    iterator end() { return recursive_directory_iterator(); }

    path p_;
};

for (auto it : recursive_directory_range(dir_path))
{
    std::cout << it << std::endl;
}

Answer 4

In C++11/14 with the "Filesystem TS", the <experimental/filesystem> header and range-for you can simply do this:

#include <experimental/filesystem>

using std::experimental::filesystem::recursive_directory_iterator;
...
for (auto& dirEntry : recursive_directory_iterator(myPath))
     cout << dirEntry << endl;

As of C++17, std::filesystem is part of the standard library and can be found in the <filesystem> header (no longer "experimental").

Answer 5

A fast solution is using C's Dirent.h library.

Working code fragment from Wikipedia:

#include <stdio.h>
#include <dirent.h>

int listdir(const char *path) {
    struct dirent *entry;
    DIR *dp;

    dp = opendir(path);
    if (dp == NULL) {
        perror("opendir: Path does not exist or could not be read.");
        return -1;
    }

    while ((entry = readdir(dp)))
        puts(entry->d_name);

    closedir(dp);
    return 0;
}

Answer 6

In addition to the above mentioned boost::filesystem you may want to examine wxWidgets::wxDir and Qt::QDir.

Both wxWidgets and Qt are open source, cross platform C++ frameworks.

wxDir provides a flexible way to traverse files recursively using Traverse() or a simpler GetAllFiles() function. As well you can implement the traversal with GetFirst() and GetNext() functions (I assume that Traverse() and GetAllFiles() are wrappers that eventually use GetFirst() and GetNext() functions).

QDir provides access to directory structures and their contents. There are several ways to traverse directories with QDir. You can iterate over the directory contents (including sub-directories) with QDirIterator that was instantiated with QDirIterator::Subdirectories flag. Another way is to use QDir's GetEntryList() function and implement a recursive traversal.

Here is sample code (taken from here # Example 8-5) that shows how to iterate over all sub directories.

#include <qapplication.h>
#include <qdir.h>
#include <iostream>

int main( int argc, char **argv )
{
    QApplication a( argc, argv );
    QDir currentDir = QDir::current();

    currentDir.setFilter( QDir::Dirs );
    QStringList entries = currentDir.entryList();
    for( QStringList::ConstIterator entry=entries.begin(); entry!=entries.end(); ++entry) 
    {
         std::cout << *entry << std::endl;
    }
    return 0;
}

Answer 7

Boost::filesystem provides recursive_directory_iterator, which is quite convenient for this task:

#include "boost/filesystem.hpp"
#include <iostream>

using namespace boost::filesystem;

recursive_directory_iterator end;
for (recursive_directory_iterator it("./"); it != end; ++it) {
    std::cout << *it << std::endl;                                    
}

Answer 8

You can use ftw(3) or nftw(3) to walk a filesystem hierarchy in C or C++ on POSIX systems.

Answer 9

You don't. The C++ standard has no concept of directories. It is up to the implementation to turn a string into a file handle. The contents of that string and what it maps to is OS dependent. Keep in mind that C++ can be used to write that OS, so it gets used at a level where asking how to iterate through a directory is not yet defined (because you are writing the directory management code).

Look at your OS API documentation for how to do this. If you need to be portable, you will have to have a bunch of #ifdefs for various OSes.

Answer 10

You would probably be best with either boost or c++14's experimental filesystem stuff. IF you are parsing an internal directory (ie. used for your program to store data after the program was closed), then make an index file that has an index of the file contents. By the way, you probably would need to use boost in the future, so if you don't have it installed, install it! Second of all, you could use a conditional compilation, e.g.:

#ifdef WINDOWS //define WINDOWS in your code to compile for windows
#endif

The code for each case is taken from https://stackoverflow.com/a/67336/7077165

#ifdef POSIX //unix, linux, etc.
#include <stdio.h>
#include <dirent.h>

int listdir(const char *path) {
    struct dirent *entry;
    DIR *dp;

    dp = opendir(path);
    if (dp == NULL) {
        perror("opendir: Path does not exist or could not be read.");
        return -1;
    }

    while ((entry = readdir(dp)))
        puts(entry->d_name);

    closedir(dp);
    return 0;
}
#endif
#ifdef WINDOWS
#include <windows.h>
#include <string>
#include <vector>
#include <stack>
#include <iostream>

using namespace std;

bool ListFiles(wstring path, wstring mask, vector<wstring>& files) {
    HANDLE hFind = INVALID_HANDLE_VALUE;
    WIN32_FIND_DATA ffd;
    wstring spec;
    stack<wstring> directories;

    directories.push(path);
    files.clear();

    while (!directories.empty()) {
        path = directories.top();
        spec = path + L"\\" + mask;
        directories.pop();

        hFind = FindFirstFile(spec.c_str(), &ffd);
        if (hFind == INVALID_HANDLE_VALUE)  {
            return false;
        } 

        do {
            if (wcscmp(ffd.cFileName, L".") != 0 && 
                wcscmp(ffd.cFileName, L"..") != 0) {
                if (ffd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
                    directories.push(path + L"\\" + ffd.cFileName);
                }
                else {
                    files.push_back(path + L"\\" + ffd.cFileName);
                }
            }
        } while (FindNextFile(hFind, &ffd) != 0);

        if (GetLastError() != ERROR_NO_MORE_FILES) {
            FindClose(hFind);
            return false;
        }

        FindClose(hFind);
        hFind = INVALID_HANDLE_VALUE;
    }

    return true;
}
#endif
//so on and so forth.

Answer 11

You need to call OS-specific functions for filesystem traversal, like open() and readdir(). The C standard does not specify any filesystem-related functions.

Answer 12

You don't. Standard C++ doesn't expose to concept of a directory. Specifically it doesn't give any way to list all the files in a directory.

A horrible hack would be to use system() calls and to parse the results. The most reasonable solution would be to use some kind of cross-platform library such as Qt or even POSIX.

Answer 13

We are in 2019. We have filesystem standard library in C++. The Filesystem library provides facilities for performing operations on file systems and their components, such as paths, regular files, and directories.

There is an important note on this link if you are considering portability issues. It says:

The filesystem library facilities may be unavailable if a hierarchical file system is not accessible to the implementation, or if it does not provide the necessary capabilities. Some features may not be available if they are not supported by the underlying file system (e.g. the FAT filesystem lacks symbolic links and forbids multiple hardlinks). In those cases, errors must be reported.

The filesystem library was originally developed as boost.filesystem, was published as the technical specification ISO/IEC TS 18822:2015, and finally merged to ISO C++ as of C++17. The boost implementation is currently available on more compilers and platforms than the C++17 library.

@adi-shavit has answered this question when it was part of std::experimental and he has updated this answer in 2017. I want to give more details about the library and show more detailed example.

std::filesystem::recursive_directory_iterator is an LegacyInputIterator that iterates over the directory_entry elements of a directory, and, recursively, over the entries of all subdirectories. The iteration order is unspecified, except that each directory entry is visited only once.

If you don't want to recursively iterate over the entries of subdirectories, then directory_iterator should be used.

Both iterators returns an object of directory_entry. directory_entry has various useful member functions like is_regular_file, is_directory, is_socket, is_symlink etc. The path() member function returns an object of std::filesystem::path and it can be used to get file extension, filename, root name.

Consider the example below. I have been using Ubuntu and compiled it over the terminal using

g++ example.cpp --std=c++17 -lstdc++fs -Wall

#include <iostream>
#include <string>
#include <filesystem>

void listFiles(std::string path)
{
    for (auto& dirEntry: std::filesystem::recursive_directory_iterator(path)) {
        if (!dirEntry.is_regular_file()) {
            std::cout << "Directory: " << dirEntry.path() << std::endl;
            continue;
        }
        std::filesystem::path file = dirEntry.path();
        std::cout << "Filename: " << file.filename() << " extension: " << file.extension() << std::endl;

    }
}

int main()
{
    listFiles("./");
    return 0;
}

Answer 14

If you are on Windows, you can use the FindFirstFile together with FindNextFile API. You can use FindFileData.dwFileAttributes to check if a given path is a file or a directory. If it's a directory, you can recursively repeat the algorithm.

Here, I have put together some code that lists all the files on a Windows machine.

http://dreams-soft.com/projects/traverse-directory