Recursive Make - friend or foe?

I'm using (GNU) Make in my project. I'm currently putting one makefile per directory and specify the subdirectories using SUBDIRS. It's been suggested to me that this is not the ideal way of using make, that using a one toplevel make file (or several, split up using include). I've tried migrating/using this layout in the past, but it appears to me that it's unnecessary complicated.

Which are the benefits/drawbacks of using recursive makefiles?

From stackoverflow

• The benefit that I've gotten from this in the past is that it's easier to build files in a single subdirectory. You can do this with dependencies, but it's a bit more work to keep all of the targets straight. Basically, this makes it easier to make changes and test one library without having to deal with the full complexity of the larger project.

  Johan Dahlin : You're right. Easier reusage of common targets is another advantage

• The issue with recursive make is the time overhead of evaluating all the different make files vs. evaluating one large make file. Part of this is just spawning processes but also (IIRC) you tend to be forced into assuming that other makes files did something and rebuilding when you don't really need to.

  My take on it is to have a single make file per "Unit", that more or less amounts to having a make file for each chunk of code that you expect could be used on it's own (e.g. as an independent library)

  OTOH my current project breaks this all over the place as I'm generating make files during the build. :b

  JesperE : No, time is not the biggest problem. The biggest problem is partitioning the dependency tree into several dependency trees, which prevents you from properly expressing dependencies across sub-makes.

  BCS : IMNSHO the only reason you even care about dependencies at all *is* time. If you don't care about time, just do a from scratch, blank slate rebuild every time. You can get "correct" dependencies expressed by coding so if anything changes, it's assumed that everything changed, but it takes more time.

• An article entitled "Recursive Make Considered Harmful" can be found here: http://miller.emu.id.au/pmiller/books/rmch/?ref=DDiyet.Com. (Or at the Aegis project at SourceForge.)

  It explores the problems with recursive makefiles, and recommends a single-makefile approach.

  Johan Dahlin : That's a good reference, thanks for pointing it out.

  Dana the Sane : Unfortunately, the link to the actual paper is giving a 500.

  caspin : The link to paper seem to be ok now.

• To throw in a third option, you could use GNU Autotools. Mostly used for other reasons, but may also helpful at organizing a multi-directory build.

  http://www.lrde.epita.fr/~adl/autotools.html

  It has to be noted, though, that the result is a recursive version.

  Johan Dahlin : I'm actually using autotools, but you can use both recursive and non-recursive mode.

• The first thing you should keep in mind (just to eliminate any misunderstanding) is that we're not talking about a single vs. multiple makefiles. Splitting your makefile in one per subdirectory is probably a good idea in any case.

  Recursive makefiles are bad primarily because you partition your dependency tree into several trees. This prevents dependencies between make instances from being expressed correctly. This also causes (parts of) the dependency tree to be recalculated multiple times, which is a performance issue in the end (although usually not a big one.)

  There are a couple of tricks you need to use in order to properly use the single-make approach, especially when you have a large code base:

  First, use GNU make (you already do, I see). GNU make has a number of features which simplifies things, and you won't have to worry about compatibilities.

  Second, use target-specific variable values. This will allow you to have, for example, different values of CFLAGS for different targets, instead of forcing you to have a single CFLAGS in your entire make:

   main: CFLAGS=-O2
   lib: CFLAGS=-O2 -g
  

  Third, make sure you use VPATH/vpath to the full extent supported by GNU make.

  You also want to make sure that you do not have multiple source files with the same name. One limitation of VPATH is that it does not allow you to have target-specific VPATH definitions, so the names of your source files will have to co-exist in a single "VPATH namespace".

  Johan Dahlin : Great answer, thanks! I'm actually using VPATH even for a project using recursive make files, it makes like a lot easier, especially for srcdir != builddir builds.

• Run, don't walk, to cmake.org and get Cmake, one of the best build tools available.

  You will still be using GNU make, but in this case CMake will generate the makefiles for you.

  I can't guarantee 100%, but I have yet to come across a case where it has not correctly handled dependencies between subdirectories correctly (ie the problem that plagues the recursive make). At the very least it is a lot easier to maintain Cmakefiles than makefiles. Highly recommended.

  Do not use GNU autotools - that way madness lies!

  Johan Dahlin : I already know GNU autotools too well and I'm getting increasingly old to learn a new system. Maybe in another 5 years :-)

  Kristopher Johnson : CMake is worth the effort needed to learn. It is much easier than Autotools, and it's cross-platform support is much better.