Checkmake: Experimental Linter/Analyzer for Makefiles(github.com)
github.com
Checkmake: Experimental Linter/Analyzer for Makefiles
https://github.com/mrtazz/checkmake
9 comments
[deleted]
Cool!
I also have an experimental (GNU) makefile auditor called "make-audit", available here: https://github.com/david-a-wheeler/make-audit
Per its README: "It will report when an execution of make reads or changes files in ways that are inconsistent with its Makefile. It requires an "auditor" tool named pmaudit in your PATH. It also requires GNU make to be installed and run as make."
Note that these tools are looking for different things and taking different approaches. My make-audit tool is a dynamic analysis tool - you run make and it watches what is done. This "checkmake" tool appears to use static analysis to check against various rules (e.g., there should be an "all"), which is fine but it's a different kind of check.
Sadly, my make-audit tool is also rather experimental. As I say in its README, "This is an extremely early version. Much needs fixing." For example:
* This doesn't properly handle grouped targets or empty commands. It should handle makefiles with their own SHELL and .ONESHELL values.
* I don't think it handles multi-line make commands exactly correctly (it's close but not quite right).
* Lots more options are needed.
* It needs a better internal test suite.
Still, make-audit does do some useful things, and it'd be awesome if others would be willing to work with me to make it reliably useful.
I also have an experimental (GNU) makefile auditor called "make-audit", available here: https://github.com/david-a-wheeler/make-audit
Per its README: "It will report when an execution of make reads or changes files in ways that are inconsistent with its Makefile. It requires an "auditor" tool named pmaudit in your PATH. It also requires GNU make to be installed and run as make."
Note that these tools are looking for different things and taking different approaches. My make-audit tool is a dynamic analysis tool - you run make and it watches what is done. This "checkmake" tool appears to use static analysis to check against various rules (e.g., there should be an "all"), which is fine but it's a different kind of check.
Sadly, my make-audit tool is also rather experimental. As I say in its README, "This is an extremely early version. Much needs fixing." For example:
* This doesn't properly handle grouped targets or empty commands. It should handle makefiles with their own SHELL and .ONESHELL values.
* I don't think it handles multi-line make commands exactly correctly (it's close but not quite right).
* Lots more options are needed.
* It needs a better internal test suite.
Still, make-audit does do some useful things, and it'd be awesome if others would be willing to work with me to make it reliably useful.
Quick self-reply:
My make-audit tool only reports a few errors, but they are very general and apply to all rules. This is probably the best way to compare the tools. Here are the make-audit reports:
* Error: Target TARGET : unreported prerequisites: SET : The make recipe for creating TARGET is reading from the prerequisites in SET, but the makefile fails to report them as dependencies. You may want to add SET to the prerequisites of TARGET.
* Error: Target TARGET : claimed but unused prerequisites: SET : The make recipe for creating TARGET claims that it depends on SET, but the items in SET were never read. You may want to remove SET from the prerequisites of TARGET.
* Error: Target TARGET : unreported target: SET The make recipe for updating TARGET also modifies the files in SET but this is not reported. The easy solution is to use GNU make's grouped targets feature that it added in version 4.3, e.g., TARGET SET &: PREREQUISITES. There are alternative solutions.
* Error: Target TARGET : unmodified reported target: SET The make recipe for updating TARGET does not appear to actually write to TARGET. You may need to add a missing command to actually modify TARGET.
My make-audit tool only reports a few errors, but they are very general and apply to all rules. This is probably the best way to compare the tools. Here are the make-audit reports:
* Error: Target TARGET : unreported prerequisites: SET : The make recipe for creating TARGET is reading from the prerequisites in SET, but the makefile fails to report them as dependencies. You may want to add SET to the prerequisites of TARGET.
* Error: Target TARGET : claimed but unused prerequisites: SET : The make recipe for creating TARGET claims that it depends on SET, but the items in SET were never read. You may want to remove SET from the prerequisites of TARGET.
* Error: Target TARGET : unreported target: SET The make recipe for updating TARGET also modifies the files in SET but this is not reported. The easy solution is to use GNU make's grouped targets feature that it added in version 4.3, e.g., TARGET SET &: PREREQUISITES. There are alternative solutions.
* Error: Target TARGET : unmodified reported target: SET The make recipe for updating TARGET does not appear to actually write to TARGET. You may need to add a missing command to actually modify TARGET.
Hmm. `--list-rules` doesn't seem to work (in the docker version anyway), and I threw it at an absolutely awful Makefile (cyclic dependencies, variable spelling errors making them empty, several other clear problems) both before and after I rewrote the whole thing...
... and it's only complaining about targets over 5 lines long (really? some of the CLI tools I'm running take 20+ lines of flags spread out for readability, I'm not going to shrink that) and one or two missing .PHONY markers (it's correct about those). There's one fewer missing .PHONY after my rewrite.
Given the list in the rules folder, kinda seems like it doesn't check enough to be useful. As much as I would truly love a brutal make checker.
... and it's only complaining about targets over 5 lines long (really? some of the CLI tools I'm running take 20+ lines of flags spread out for readability, I'm not going to shrink that) and one or two missing .PHONY markers (it's correct about those). There's one fewer missing .PHONY after my rewrite.
Given the list in the rules folder, kinda seems like it doesn't check enough to be useful. As much as I would truly love a brutal make checker.
I note the maintainer has asked for help with the project and called for additional maintainers, so if you use this or want it to improve, please get involved:
https://github.com/mrtazz/checkmake/issues/44
https://github.com/mrtazz/checkmake/issues/44
Nice idea, but...
This checks only 4 rules, not even correct.
It's also implemented in Go, which is really unsuitable for a program with such a goal.
This checks only 4 rules, not even correct.
It's also implemented in Go, which is really unsuitable for a program with such a goal.
I wonder how hard it would be to modify shellcheck to work for Makefiles.
Some discussion on that here:
https://github.com/koalaman/shellcheck/issues/58
The hadolint project does shell checking for Dockerfiles and it uses shellcheck:
https://github.com/hadolint/hadolint
So the approach is definitely feasible, but you do need a new project and probably it needs to be written in Haskell.
https://github.com/koalaman/shellcheck/issues/58
The hadolint project does shell checking for Dockerfiles and it uses shellcheck:
https://github.com/hadolint/hadolint
So the approach is definitely feasible, but you do need a new project and probably it needs to be written in Haskell.
Do we still need makefiles :(? What’s the alternative?
The question isn't "do we need makefiles". The question is, what is the problem you're trying to solve?
There are lots of cases where make and makefiles are a great solution.
A few pointers though:
* In most cases, just use GNU make, don't try to stick to just POSIX make. POSIX make is too feature-poor for many situations.
* If you have more than a few files, automatically generate the dependencies (e.g., in deps/DIRS/FILE.d files). In practice that often means you need to use GNU make (or automake). Doing that by hand for a few files is fine, but if you try to scale that up, it will become a large always-wrong mess. It's pretty easy to do once you know how - it's just a few lines - but it will save endless headaches. Many compilers already have built-in support for make, so in many cases make is the easy path.
* Don't use recursive make (re-execute make in each subdirectory). Run "make" once, and give it all the correct info. It seems like this would be slow, but it's faster & easier to maintain. I routinely get executions of 0.07 seconds on 1000 files when there's no work to do (a really tiny overhead).
There are definitely alternatives. I routinely use rake, which calls ruby instead of shell as the command language. Rake can be quite useful, but I don't think it's an automatic win over make at all.
There are lots of cases where make and makefiles are a great solution.
A few pointers though:
* In most cases, just use GNU make, don't try to stick to just POSIX make. POSIX make is too feature-poor for many situations.
* If you have more than a few files, automatically generate the dependencies (e.g., in deps/DIRS/FILE.d files). In practice that often means you need to use GNU make (or automake). Doing that by hand for a few files is fine, but if you try to scale that up, it will become a large always-wrong mess. It's pretty easy to do once you know how - it's just a few lines - but it will save endless headaches. Many compilers already have built-in support for make, so in many cases make is the easy path.
* Don't use recursive make (re-execute make in each subdirectory). Run "make" once, and give it all the correct info. It seems like this would be slow, but it's faster & easier to maintain. I routinely get executions of 0.07 seconds on 1000 files when there's no work to do (a really tiny overhead).
There are definitely alternatives. I routinely use rake, which calls ruby instead of shell as the command language. Rake can be quite useful, but I don't think it's an automatic win over make at all.
I have used make extensively for two decades, and every year I come to love it more. It doesn't work well with some language designs, like Java, but Java is its own world and doesn't want to play with anything else, so that's fine.
I use it a lot to make one-off data analyses somewhat reproducible. To make them fully reproducible I use docker run for each command invocation, but sometimes I skip that step. Something like airflow is way too verbose and clunky compared to make for my use cases.
I use it a lot to make one-off data analyses somewhat reproducible. To make them fully reproducible I use docker run for each command invocation, but sometimes I skip that step. Something like airflow is way too verbose and clunky compared to make for my use cases.
I still use makefiles for a wrapper around the build steps. How else do you handle the magic incantation for
aws ecr get-login-password --region region | docker login --username AWS --password-stdin aws_account_id.dkr.ecr.region.amazonaws.com
docker push <...>
No suggestions on better alternatives though unfortunately
No suggestions on better alternatives though unfortunately
I've found Earthly[0] to be a good alternative. Its syntax is reminiscent of both Make and Dockerfiles.
I still heavily use Makefiles at my workplace. It seems to be common with Go projects.
[0]: https://earthly.dev/
I still heavily use Makefiles at my workplace. It seems to be common with Go projects.
[0]: https://earthly.dev/
> Do we still need makefiles :(?
Yes, because
> What’s the alternative?
There are endless alternatives - none of which are even remotely as ubiquitous, most of which are more complex, less portable, make aggressive tradeoffs, aren't as flexible, etc. Make is everywhere and provides a decent baseline.
Yes, because
> What’s the alternative?
There are endless alternatives - none of which are even remotely as ubiquitous, most of which are more complex, less portable, make aggressive tradeoffs, aren't as flexible, etc. Make is everywhere and provides a decent baseline.
> less portable
Makefiles being portable is an illusion. Make itself has multiple flavors. Running standard POSIX utilities like `find` or `sed` will have different outputs depending on if you're on macOS, or regular Linux. There are so many implicit dependencies that aren't codified.
It's true that Makefiles are often portable. I'm always very happy when a GitHub repo has a Makefile and I can do something like `make install` to have a binary on my system. This is because my system happens to have the required compilers and libraries, but there have also been plenty of times where I needed to install something without it being clear.
Makefiles being portable is an illusion. Make itself has multiple flavors. Running standard POSIX utilities like `find` or `sed` will have different outputs depending on if you're on macOS, or regular Linux. There are so many implicit dependencies that aren't codified.
It's true that Makefiles are often portable. I'm always very happy when a GitHub repo has a Makefile and I can do something like `make install` to have a binary on my system. This is because my system happens to have the required compilers and libraries, but there have also been plenty of times where I needed to install something without it being clear.
[0]: https://github.com/mrtazz/checkmake/tree/main/rules
[1]: https://github.com/mrtazz/checkmake/blob/main/rules/maxbodyl...