make wildcard subdirectory targets

Question

I have a "lib" directory in my applications main directory, which contains an arbitrary number of subdirectories, each having its own Makefile.

I would like to have a single Makefile in the main directory, that calls each subdirectory's Makefile. I know this is possible if I manually list the subdirs, but I would like to have it done automatically.

I was thinking of something like the following, but it obviously does not work. Note that I also have clean, test, etc. targets, so % is probably not a good idea at all.

LIBS=lib/*

all: $(LIBS)

%:
  (cd $@; $(MAKE))

Any help is appreciated!

Answer 1

The following will work with GNU make:

LIBS=$(wildcard lib/*)
all: $(LIBS)
.PHONY: force
$(LIBS): force
  cd $@ && pwd

If there might be something other than directories in lib, you could alternatively use:

LIBS=$(shell find lib -type d)

To address the multiple targets issue, you can build special targets for each directory, then strip off the prefix for the sub-build:

LIBS=$(wildcard lib/*)
clean_LIBS=$(addprefix clean_,$(LIBS))
all: $(LIBS)
clean: $(clean_LIBS)
.PHONY: force
$(LIBS): force
  echo make -C $@
$(clean_LIBS): force
  echo make -C $(patsubst clean_%,%,$@) clean

Answer 2

There is also a way of listing sub-directories with gmake commands only, without using any shell commands:

test:
  @echo $(filter %/, $(wildcard lib/*/))

This will list all sub-directories with trailing '/'. To remove it you can use the substitute pattern:

subdirs = $(filter %/, $(wildcard lib/*/))
test:
  @echo $(subdirs:%/=%)

Then to actually create rules executing makefiles in each sub-directory you can use a small trick - a phony target in a non-existent directory. I think in this case an example will tell more than any explanation:

FULL_DIRS =$(filter %/, $(wildcard lib/*/))
LIB_DIRS  =$(FULL_DIRS:%/=%)
DIRS_CMD  =$(foreach subdir, $(LIB_DIRS), make-rule/$(subdir))

make-rule/%:
  cd $* && $(MAKE)

all: DIRS_CMD

Basically, target 'all' lists all sub-directories as prerequisites. For example, if LIB_DIRS contained lib/folder1 lib/folder2 then the expansion would look like this:

all: make-rule/lib/folder1 make-rule/lib/folder2

Then 'make', in order to execute rule 'all', tries to match each prerequisite with an existing target. In this case the target is 'make-rule/%:', which uses '$*' to extract the string after 'make-rule/' and uses it as argument in the recipe. For example, the first prerequisite would be matched and expanded like this:

make-rule/lib/folder1:
  cd lib/folder1 && $(MAKE)

Answer 3

What if you want to call different targets than all in an unknown number of subdirectories?

The following Makefile uses macros so create a forwarding dummy-target for a number of subdirectories to apply the given target from the command line to each of them:

# all direct directories of this dir. uses "-printf" to get rid of the "./"
DIRS=$(shell find . -maxdepth 1 -mindepth 1 -type d -not -name ".*" -printf '%P\n')
# "all" target is there by default, same logic as via the macro
all: $(DIRS)

$(DIRS):
    $(MAKE) -C $@
.PHONY: $(DIRS)

# if explcit targets where given: use them in the macro down below. each target will be delivered to each subdirectory contained in $(DIRS).
EXTRA_TARGETS=$(MAKECMDGOALS)

define RECURSIVE_MAKE_WITH_TARGET
# create new variable, with the name of the target as prefix. it holds all
# subdirectories with the target as suffix
$(1)_DIRS=$$(addprefix $(1)_,$$(DIRS))

# create new target with the variable holding all the subdirectories+suffix as
# prerequisite
$(1): $$($1_DIRS)

# use list to create target to fullfill prerequisite. the rule is to call
# recursive make into the subdir with the target
$$($(1)_DIRS):
      $$(MAKE) -C $$(patsubst $(1)_%,%,$$@) $(1)

# and make all targets .PHONY
.PHONY: $$($(1)_DIRS)
endef

# evaluate the macro for all given list of targets
$(foreach t,$(EXTRA_TARGETS),$(eval $(call RECURSIVE_MAKE_WITH_TARGET,$(t))))

Hope this helps. Really helpfull when dealing with paralelism: make -j12 clean all in a tree with makefiles having these targets... As always: playing with make is dangerous, different meta-levels of programming are too close together ,-)