GNU Make in Detail for Beginners(linuxforu.com)
linuxforu.com
GNU Make in Detail for Beginners
http://www.linuxforu.com/2012/06/gnu-make-in-detail-for-beginners/
36 comments
A digression but .. CM Crossroads has gone downhill for a while now.
Recently it took three or four emails from me for them to remove some spam hacked into their site.
I've used gmsl in the past. It's good stuff.
Thank you for a good article. Even I learned some new stuff. However, there was a major omission: automatic dependency generation. Every now and then I forget how it's done and try to find a good source on the net, but they're really scarce. On the other hand, there are tons of small "your first makefile" tutorials (this article is better than the average).
Here's my Makefile boilerplate. It build single executable from all .c files in this directory and scans for include file dependencies to properly cause recompilation when a header file is touched.
Here's my Makefile boilerplate. It build single executable from all .c files in this directory and scans for include file dependencies to properly cause recompilation when a header file is touched.
INCS=-I./
LIBS=-lglfw -lGLEW
CFLAGS=-std=gnu99 -g -ggdb -W -Wall -Wextra -pedantic
CFLAGS+=$(INCS)
CFLAGS+=-march=native -mno-80387 -mfpmath=sse -O3
SRCS=$(wildcard *.c)
OBJS=$(SRCS:.c=.o)
DEPS=$(SRCS:.c=.d)
EXECUTABLE=main
.PHONY: all clean
all: $(EXECUTABLE)
$(EXECUTABLE): $(OBJS)
$(CC) $(CFLAGS) $(LIBS) -o $@ $^
%.o: %.c
$(CC) $(CFLAGS) -c -o $@ $<
%.d: %.c
$(CC) $(CFLAGS) -MM -o $@ $<
-include $(DEPS)
clean:
rm -f $(EXECUTABLE)
rm -f $(OBJS) $(DEPS)
A quick note:
> INCS=-I./
This is generally a bad idea, unless you're explicitly trying to override #include <stdio.h> with a local file (and there are better ways to do that). <> are for system files, "" are for local files.
So off the bat, your makefile example is better than most makefiles out there. I still see things that look like some dumb IDE wrote them with explicit .d and x.c: x.o rules littered about.
Though I do note you aren't taking advantage of the built-in makefile rules (which are defined in POSIX), and you aren't taking advantage of gcc's -MMD flag, which does the dependencies during compilation instead of a separate step. That eliminates much of the file:
> INCS=-I./
This is generally a bad idea, unless you're explicitly trying to override #include <stdio.h> with a local file (and there are better ways to do that). <> are for system files, "" are for local files.
So off the bat, your makefile example is better than most makefiles out there. I still see things that look like some dumb IDE wrote them with explicit .d and x.c: x.o rules littered about.
Though I do note you aren't taking advantage of the built-in makefile rules (which are defined in POSIX), and you aren't taking advantage of gcc's -MMD flag, which does the dependencies during compilation instead of a separate step. That eliminates much of the file:
LDLIBS=-lglfw -lGLEW
CFLAGS+=-std=gnu99 -g -ggdb -W -Wall -Wextra -pedantic
CFLAGS+=-MMD
CFLAGS+=$(INCS)
CFLAGS+=-march=native -mno-80387 -mfpmath=sse -O3
SRCS=$(wildcard *.c)
OBJS=$(SRCS:.c=.o)
EXECUTABLE=main
.PHONY: all clean
all: $(EXECUTABLE)
$(EXECUTABLE): $(OBJS)
$(CC) $(CFLAGS) $(LDLIBS) -o $@ $^
-include *.d
clean:
rm -f $(EXECUTABLE)
rm -f $(OBJS) *.d
LDLIBS=-lglfw -lGLEW
CFLAGS+=-std=gnu99 -g -ggdb -W -Wall -Wextra -pedantic
CFLAGS+=-MMD
CFLAGS+=$(INCS)
CFLAGS+=-march=native -mno-80387 -mfpmath=sse -O3
OBJS=main.o obj1.o obj2.o
.PHONY: all clean
all: main
main: $(OBJS)
-include *.d
clean:
rm -f $(EXECUTABLE)
rm -f $(OBJS) *.d
.PHONY: all clean
all: main
main: LDLIBS=-lglfw -lGLEW
main: CFLAGS+=-std=gnu99 -g -ggdb -W -Wall -Wextra -pedantic
main: CFLAGS+=-MMD
main: CFLAGS+=$(INCS)
main: CFLAGS+=-march=native -mno-80387 -mfpmath=sse -O3
main: OBJS=main.o obj1.o obj2.o
main: $(OBJS)
-include *.d
clean:
rm -f $(EXECUTABLE)
rm -f $(OBJS) *.d
I don't know about out-of-source building, but I know Make has a bunch of built-in rules that you can use to cut down your boilerplate.
For example, I'm pretty sure that a filename depending on objects will automatically be created by linking those objects (although you'd have to specify your libraries in LDFLAGS rather than LIBS. Also, your rule for building .o files from .c files would be automatic if you put your include-directives in CPPFLAGS instead of INCS.
For example, I'm pretty sure that a filename depending on objects will automatically be created by linking those objects (although you'd have to specify your libraries in LDFLAGS rather than LIBS. Also, your rule for building .o files from .c files would be automatic if you put your include-directives in CPPFLAGS instead of INCS.
You might want to provide separate CFLAGS and LDFLAGS.
Also, I prefer adding -MMD to CFLAGS instead of having an additional rule for dependency generation: This way, you won't unnecessarily trigger dependency generation on targets which do not build objects like clean.
Also, I prefer adding -MMD to CFLAGS instead of having an additional rule for dependency generation: This way, you won't unnecessarily trigger dependency generation on targets which do not build objects like clean.
(Sorry, I misread the above post, but I believe LDLIBS is the Makefile built-in variable for libraries to link.)
Actually, in this case, I believe LDLIBS would be more appropriate since the OP isn't passing any flags to `ld` but rather the libraries for `ld` to link. I ended up getting hit be this in a simple Makefile a few weeks ago that worked in Fedora but failed in Ubuntu.
The target was "%.o: %.c", which make auto-expands to be (something like):
Actually, in this case, I believe LDLIBS would be more appropriate since the OP isn't passing any flags to `ld` but rather the libraries for `ld` to link. I ended up getting hit be this in a simple Makefile a few weeks ago that worked in Fedora but failed in Ubuntu.
The target was "%.o: %.c", which make auto-expands to be (something like):
$(CC) $(CFLAGS) $(LDFLAGS) $@ $^ $(LDLIBS)
I was more concerned about passing CFLAGS when linking - he already has a LIBS variable; you're right that the 'canonical' name for his LIBS variable is LDLIBS (as well as LOADLIBES - anyone an idea where that name came from?).
EDIT: stop editing your answer while I'm writing my own ;)
According to the manual, the built-in rule for linking an executable `n` from a single object file is
EDIT: stop editing your answer while I'm writing my own ;)
According to the manual, the built-in rule for linking an executable `n` from a single object file is
$(CC) $(LDFLAGS) n.o $(LOADLIBES) $(LDLIBS)Here's a makefile: http://code.google.com/p/min-game/source/browse/makefile
Here's the dependency script: http://code.google.com/p/min-game/source/browse/dep.sh
Both inspired by code in "Recursive Make Considered Harmful", always worth a read: http://aegis.sourceforge.net/auug97.pdf
The built-in rules and variables for gnu make are worthless for anything but the most trivial project; don't bother with them. I'd be so happy if there was a makefile directive that forced make to run in "make -rR" mode.
Here's the dependency script: http://code.google.com/p/min-game/source/browse/dep.sh
Both inspired by code in "Recursive Make Considered Harmful", always worth a read: http://aegis.sourceforge.net/auug97.pdf
The built-in rules and variables for gnu make are worthless for anything but the most trivial project; don't bother with them. I'd be so happy if there was a makefile directive that forced make to run in "make -rR" mode.
You should put something like this:
ifneq ($(MAKECMDGOALS),clean)
include $(DEPS)
endif
I used this (on an admittedly unfinished and more or less abandoned project) https://github.com/adam-a/Simple-Sampler/blob/master/build/M...
I never found a good way to silo the .o and .d files though, the way I do it is put the main makefile in a separate dir from the tree so all the clutter can accumulate there.
I never found a good way to silo the .o and .d files though, the way I do it is put the main makefile in a separate dir from the tree so all the clutter can accumulate there.
Fair warning: GNU make cannot handle directory names or file names which contain spaces. This is a show-stopper defect which has been present since the day it was written and shows no signs of ever being fixed.
Why would you have a space in a directory/file name though? I wouldn't call this a show-stopper.
Program Files
My Documents
Also, why wouldn't I? Why shouldn't I? There isn't a single file system that prevents me from doing so and there isn't a single other computer program in the world which has a problem with it.
My Documents
Also, why wouldn't I? Why shouldn't I? There isn't a single file system that prevents me from doing so and there isn't a single other computer program in the world which has a problem with it.
Make was made for *nix, it's not exactly surprising that it doesn't conform with Windows's preferences :-) I certainly wouldn't call that show-stopping.
Oh and there are some file systems that don't allow spaces by the way, though admittedly not in common usage.
Oh and there are some file systems that don't allow spaces by the way, though admittedly not in common usage.
I understand you are making a point, one which I largely agree with, but:
there isn't a single other computer program in the world which has a problem with it
is a bit too much hyperbole -- there is some really really bad software out there.
there isn't a single other computer program in the world which has a problem with it
is a bit too much hyperbole -- there is some really really bad software out there.
That's a fair point. Strawberry Perl, I'm looking at you.
I stand by the rest of what I said, though. If a piece of software can't handle spaces in paths, the software is broken. I'm not adding a top-level directory to my C: drive just to appease it. I'll use something else.
I stand by the rest of what I said, though. If a piece of software can't handle spaces in paths, the software is broken. I'm not adding a top-level directory to my C: drive just to appease it. I'll use something else.
Sigh, yet another article giving advice about how to use make as a kind of general-purpose language. It's not. Forget all the crap with wildcards, conditionals, filename globbing, etc. and use it for what it is: a program that takes as input a DAG and a timestamp for each node, and executes rules when predecessor(s) of some node are newer than the node itself.
Take make for what it is and use a proper programming language to handle Makefile generation, installation, etc.
Take make for what it is and use a proper programming language to handle Makefile generation, installation, etc.
c:\>copy con m.bat
cl foo.c bar.c baz.c boz.c
^ZI really appreciate that this article talks about using Make in contexts other than just compiling software. I've used it in the past to automate bioinformatics analysis pipelines. Unfortunately, I ultimately found that it did not really have the flexibility that I needed for more complex analyses, because of various limitations such as the inability to handle multiple output files from a single rule. I also couldn't figure out how to do automatic dependency generation in the general case (not just for C files).
I figure you already solved that problem, but maybe http://gittup.org/tup/ is general enough for your use case?
No, I haven't solved that problem. tup looks really useful. Thanks for the suggestion!
It's worth highlighting here how Tup differs fundamentally from Make in that you specify the DAG from the bottom-up, rather than the top-down as in Make.
That is to say, with the top-down scheme used by Make you think about and write things down in this kind of order:
To put it another way, Make is declarative like a functional language, whereas Tup more like an imperative language. If you're like me, building is a chore and a build system is supposed to make my life easier, so if it's very much more difficult than a shell script then I can't be bothered with it. Like most programmers I'm more at home with imperative languages than functional; and looking at other people's Makefiles tends to do my head in.
Having said that while Tup has less cruft and some better features (eg. multiple output files as per the parent poster's wish) than Make, it's still a domain-specific language and as such its overall features are limited to what its creator decided were sufficient to his definition of "building programs" for the sake of terser syntax. So, if you're like me you may prefer to hurry on to the 'ultimate' step which is to just use a general-purpose language with a helper library for running build commands. I currently use fabricate.py (http://code.google.com/p/fabricate/) which provides a 'run("shell command")' function to a Python script that runs the command while hooking system calls to spy on the command's inputs and outputs, then saves the command-line and the input/output file names to a file ('.deps') so next time it can check whether the command needs to be re-run or not.
Of course in a general-purpose language such as Python you can do just about anything you like and don't have problems with silly little things like spaces in strings (though see below). While the resulting script may not be as short as the 'ideal' Makefile or Tupfile which could be nearly empty with all their implicit rules taken into account, I'm of the view that explicit is better than implicit.
Couple of specific fabricate.py caveats - being on Windows, I had to do some hackery on my copy of the library to make the dependency detection work better, and to allow for supplying the dependency filenames manually as a fallback. And, the documentation of that 'run()' function could be better (in short, you shouldn't just supply a full string as you would type it at the command line; you should convert those space-seperated command-line arguments into comma-seperated Python arguments, and Fabricate will automatically quote any argument that contains a space. I have no connection with the project but I should really see if I can get into their wiki there and add a little doc laying out the subtleties of it; it does work out in the end). That said, I couldn't be much happier now; it works like a champ everywhere I've tried it including C++, Java, ActionScript and Haxe.
The other downside of bottom-up/imperative style is that it doesn't lend itself automatically to parallel builds (cf. an oft-noted difference between imperative and functional languages in general). I hear it's possible if you plan for it (http://code.google.com/p/fabricate/wiki/ParallelBuilding) though personally my projects have not been big enough to bother trying this so far.
That is to say, with the top-down scheme used by Make you think about and write things down in this kind of order:
1. executable <- object files
2. object files <- source files
1. source files -> object files
2. object files -> executable
To put it another way, Make is declarative like a functional language, whereas Tup more like an imperative language. If you're like me, building is a chore and a build system is supposed to make my life easier, so if it's very much more difficult than a shell script then I can't be bothered with it. Like most programmers I'm more at home with imperative languages than functional; and looking at other people's Makefiles tends to do my head in.
Having said that while Tup has less cruft and some better features (eg. multiple output files as per the parent poster's wish) than Make, it's still a domain-specific language and as such its overall features are limited to what its creator decided were sufficient to his definition of "building programs" for the sake of terser syntax. So, if you're like me you may prefer to hurry on to the 'ultimate' step which is to just use a general-purpose language with a helper library for running build commands. I currently use fabricate.py (http://code.google.com/p/fabricate/) which provides a 'run("shell command")' function to a Python script that runs the command while hooking system calls to spy on the command's inputs and outputs, then saves the command-line and the input/output file names to a file ('.deps') so next time it can check whether the command needs to be re-run or not.
Of course in a general-purpose language such as Python you can do just about anything you like and don't have problems with silly little things like spaces in strings (though see below). While the resulting script may not be as short as the 'ideal' Makefile or Tupfile which could be nearly empty with all their implicit rules taken into account, I'm of the view that explicit is better than implicit.
Couple of specific fabricate.py caveats - being on Windows, I had to do some hackery on my copy of the library to make the dependency detection work better, and to allow for supplying the dependency filenames manually as a fallback. And, the documentation of that 'run()' function could be better (in short, you shouldn't just supply a full string as you would type it at the command line; you should convert those space-seperated command-line arguments into comma-seperated Python arguments, and Fabricate will automatically quote any argument that contains a space. I have no connection with the project but I should really see if I can get into their wiki there and add a little doc laying out the subtleties of it; it does work out in the end). That said, I couldn't be much happier now; it works like a champ everywhere I've tried it including C++, Java, ActionScript and Haxe.
The other downside of bottom-up/imperative style is that it doesn't lend itself automatically to parallel builds (cf. an oft-noted difference between imperative and functional languages in general). I hear it's possible if you plan for it (http://code.google.com/p/fabricate/wiki/ParallelBuilding) though personally my projects have not been big enough to bother trying this so far.
Thanks for another interesting suggestion. Reading the fabricate "how it works" page, I see that you can subclass it to implement your own dependency-determination code, which is cool.
A rule for multiple output files:
output1 output2: depsYes, you can do that, but it doesn't actually work the way it should. I don't remember the details any more, but it breaks in nonintuitive ways.
Great article! I could have only wished for such fine introductory material when I was starting off with Make. It would be nice if the docs themselves had a similar chapter.
One thing that I've always wanted from Make was a way of writing long commands with less escaping and without continuation characters. Something like heredoc for Make.
For example, the following is just too arcane and I would by all means move the body of the command into a separate file:
One thing that I've always wanted from Make was a way of writing long commands with less escaping and without continuation characters. Something like heredoc for Make.
For example, the following is just too arcane and I would by all means move the body of the command into a separate file:
compress:
find -iname '*.html' -print0 | \
while IFS="" read -r -d "" fn; do \
old=$$(stat -c "%s" "$$fn"); \
java -jar htmlcompressor.jar "$$fn" -o "$$fn"; \
new=$$(stat -c "%s" "$$fn"); \
delta=$$(echo "scale=2 ; 100 - ($$new/$$old) * 100" | bc); \
echo "compressed $${fn}: $${old}/$${new} $${delta}%"; \
done;
compress:
%cmd -flags -that -control -escaping << EOF
find -iname '*.html' -print0 | while IFS="" read -r -d "" fn; do
old=$(stat -c "%s" "${fn}")
java -jar htmlcompressor.jar "${fn}" -o "${fn}"
new=$(stat -c "%s" "${fn}")
delta=$(echo "scale=2 ; 100 - (${new}/${old}) * 100" | bc)
echo "compressed ${fn}: ${old}/${new} ${delta}%"
done
EOF
This paper provides really valuable insights about Make's /modus operandi/ and desired usage:
http://miller.emu.id.au/pmiller/books/rmch/
http://miller.emu.id.au/pmiller/books/rmch/
I used to be a pretty good GNU Make 'expert'. I must admit that I'm glad that I've been lucky enough to be able to purge this arcane and esoteric knowledge from my head. Long live .PHONY!
Is recursive make not considered harmful, i.e. best practice is to instead include makefiles from subdirectories ( http://aegis.sourceforge.net/auug97.pdf )?
Including makefiles from subdirectories is not recursive make. Recursive make is reinvoking make on a subdirectory.
Ehhh, that's what I said? In the article he takes about reinvoking make using $(MAKE), having mentioned changing to a subdirectory in the previous paragraph.
It would be nice to Make tutorials to mention the simplest possible Makefile ... none at all.
1. Create something.c
2. make something
Look, no Makefile!
1. Create something.c
2. make something
Look, no Makefile!
What about when?
"Large projects can contain thousands of lines of code, distributed in multiple source files, written by many developers and arranged in several subdirectories."
"Large projects can contain thousands of lines of code, distributed in multiple source files, written by many developers and arranged in several subdirectories."
Assuming you have GNU Make installed, go to your terminal and run `info make'. Quite a read but comprehensive.
http://www.lulu.com/shop/john-graham-cumming/gnu-make-unleas...
If you don't like the book format then the original articles (which I cleaned up and improved for the book) are here:
http://blog.jgc.org/2010/01/update-list-of-my-gnu-make-artic...
Also, I created a useful 'standard library' of functions for GNU Make which can be found here: http://gmsl.sf.net/