C++ stream object threadsafe?

Question

I need to provide text searching in files in a folder in C++ application with multiple threads. If two threads try to open same file in read only mode will it lead to data races or multiple threads can read files using stream objects.?

  std::ifstream ifs ("test.txt");
  std::filebuf* inbuf  = ifs.rdbuf();

Also the files in the folder are dynamic (someone can put the files in folder while application is running) so if locking is required should I keep it on folder level or for every file lock is required dynamically?

Answer 1

Multiple threads can open the same file, and read it. This is no problem, but threads must not share std::ifstream instances. Thus, you code can be executed by multiple threads.

If the files are dynamic, I guess that your application "consume" the files, nop ? If it is the case, you must synchronize your threads: one thread must list files in the directory, and allocate files to the other threads, that read files. If all threads read files without communication between them, they will read / consume multiple times a same file.

For the lock, I don't know: it depends on what you are doing... delete files ? write in these files ? rename them ? If you need some performance, one lock for each file is better.

Answer 2

If you use the same ifstream, then you have data races (there is no locking within a stream object, according to the standard)

If you read using different streams, then it's safe.

I'm doubting that using multiple threads reading files on disk will give you any benefit, however. It has been shown (several times) that reading files in multiple threads is much slower than than reading from a single thread (assuming your "search" is reasonably efficient, of course).