How to debug parallel scons builts

Question

I have a rather complex SConstruct file and set up, so pasting it here is impractical. When I run scons -j 8 the build fails with some C++ errors. However, when I run the same clean build with just scons (so, no parallel built) then the compile finishes fine. I think that I have some dependencies problem that only appear when the code is compiled in parallel. The build takes a long time so I would like to fix this problem.

How can I debug scons so that it tells me where the problem lies?

Answer 1

I recently had similar problems in my build system at work. The first problem I encountered was very strange, as it seemed like SCons was completely messing up the directory it was supposed to be in. Upon consulting the SCons user's mailing list, I was told that you cant change directories with os.system('cd ...'), or similar while building in parallel. I wasnt doing anything like that, but upon further inspection found that the Python shutil.make_archive() function I was using in a custom builder was indeed changing directories. When I changed that, the parallel build worked much better. So, if you have any custom builders, etc make sure they are not changing the directory.

I still had problems though, similar to yours where the dependency system didnt seem to be working correctly while building in parallel. I later realized that it was my fault as the dependencies weren't as I originally expected. You can use the scons --debug=tree command-line option to help debugging the dependencies.

I originally thought that SCons had bugs in its parallel build system, but later found that its actually quite robust, and the errors I had in my build scripts made perfect sense that the parallel build would break.