Understand Makefile with two-step dependency calculation

Question

I wanted to build some C++ code. I've already had a magic Makefile that could calculate dependencies automaticaly, but I wanted to keep object files in separate directory. So the structure looks like:

• Makefile
• source/a/
• source/b/
• source/c/
• obj/

I was able to run Makefile that converted sources into objects using obj/ prefix, but things got messy when I wanted to backport dependency calculation.

The problem is that is almost all Makefile examples this is done in one step, creating .d from .cpp (%.d: %.cpp). I spent some time trying to understand Makefile and fix it with trials and errors. Finally I've got a working script:

DIRS := a b c
SOURCES := $(foreach dir,$(DIRS),$(wildcard source/$(dir)/*.cpp))
OBJECTS := $(patsubst %.cpp,obj/%.o,$(SOURCES))

obj/%.d: %.cpp
    @mkdir -p $(@D)
    @$(CXX) -E $(CXXFLAGS) -MM -MP -MF $@ -MQ $(@:.d=.o) -MQ $@ $<

obj/%.o: %.cpp obj/%.d
    @$(CXX) $(CXXFLAGS) -o $@ -c $<

project: $(OBJECTS)
    ...

-include $(OBJECTS:.o=.d)

But I'm still not sure why do I need to have separate steps for cpp->d and cpp&d -> o (or maybe I understand - but then I have no idea why most examples don't need this).

Answer 1

Suppose the source files are foo.cpp and bar.cpp, and the headers are foo.h and bar.h, such that foo.cpp includes foo.h and bar.h, and bar.cpp includes bar.h.

You can have a simple makefile that does not involve dependency files at all:

project: $(OBJECTS)
    ...

%.o: %.cpp
    ...

Now if you modify foo.cpp, Make will rebuild foo.o (and project). If you modify bar.cpp, Make will rebuild bar.o (and project). If you modify a header, Make will do nothing, because Make doesn't know that the objects depend in any way upon the headers.

You can put in a rule for dependency files:

project: $(OBJECTS)
    ...

%.o: %.cpp
    ...

%.d:
    ...

-include *.d

Now Make will build and use the dependency files-- if you remember to tell it to. If you do that, Make will know that when you modify foo.h it must rebuild foo.o, and when you modify bar.h it must rebuild foo.o and bar.o.

Remembering to rebuild dependency files is a pain. It would be better to let Make do it. When must foo.d be rebuilt? Well, when foo.cpp is modified, certainly:

project: $(OBJECTS)
    ...

%.o: %.cpp
    ...

%.d: %.cpp
    ...

-include *.d

But foo.d must also be rebuilt when any of the other prerequisites of foo.o (i.e. foo.h or bar.h) has been modified. I haven't spelled out the commands in the rules (and I can't make much sense of the command in your %.d rule), but a simple rule will build a foo.d that looks like this:

foo.o: foo.cpp foo.h bar.h

It is possible to write a command that will produce a foo.d that looks like this:

foo.o: foo.cpp foo.h bar.h

foo.d: foo.cpp foo.h bar.h

There are more sophisticated methods, but this is enough for today.

As to why many example makefiles share a certain sub-optimal feature, it's because people copy each others' makefiles without rigorously examining or understanding every feature.