top | item 32541016

An opinionated approach to GNU Make

185 points| hasheddan | 3 years ago |tech.davis-hansson.com | reply

185 comments

order
[+] bee_rider|3 years ago|reply
I like parts of it!

The tabs vs spaces thing seems pretty silly to me. If your editor is randomly swapping tabs and spaces, get a better editor. Tab is the default in a makefile and that seems fine. The suggestion to use "> " instead of tab just looks noisier.

The observation about the filesystem is good and hopefully well known. The way to think about makefiles is as a tool for creating files (it is very oriented toward this), not as a general purpose scripting language (it is just a worse version of whatever scripting language you are in it, if you use it this way). I do wonder if he could structure his tests to have them actually generate output files which make could track, and also have his tests track dependencies.

Point taken about the magic variables. Sometimes they can get obscure (although they are pretty easy to look up). IMO one he's missing, though, is the pattern matching % operator. If make isn't generating at least some of your recipes for you, then why not just make a "build.sh" script?

[+] bin_bash|3 years ago|reply
I think the `.RECIPEPREFIX = >` bit triggered a lot of people here in the comments, and I agree. That would make drafting newlines a huge pain in any editor. Just enable "Show Whitespace" in your editor if you want this.

That said, I'm more concerned about the guidance to not use .PHONY and instead do this:

    # Tests - re-ran if any file under src has been changed since tmp/.tests-passed.sentinel was last touched
    tmp/.tests-passed.sentinel: $(shell find src -type f)
    > mkdir -p $(@D)
    > node run test
    > touch $@ 
The author is right, that does use make in a more more idiomatic way by relying on a real file, but I see 2 major problems:

* That's a lot of logic for something that should just be super simple.

* When I say `make test` I want it to run the tests. I don't care if they've passed before and the files haven't changed.

Really though, make just isn't a great tool for build scripts. The syntax is horrific and it's hard to scale it into something readable.

If I started a new project I'd probably consider Just: https://github.com/casey/just (though I haven't had a chance to use it myself yet).

[+] brabel|3 years ago|reply
> I don't care if they've passed before and the files haven't changed.

This is a common interjection from some people... which means you think that your tests are not deterministic, otherwise it would be completely pointless to run them again without inputs changing.

I actually admit that from end-to-end tests, this is usually true despite our best efforts to the contrary... but for unit tests, I really think tests should be 100% deterministic and only ever run when something they rely on changes. The Unison programming language goes even further[1] and it NEVER executes a test again once the code under test has been "committed" into its image.

[1] https://www.unison-lang.org/

[+] kennu|3 years ago|reply
> .RECIPEPREFIX = >

That seems like a terrible idea. You change the basic syntax of the entire Makefile, forcing anybody reading it to get used to your custom indentation, where almost every line starts with an unnecessary >.

[+] nrclark|3 years ago|reply
This article has some questionable advice imo.

    SHELL := bash
Bash is a much slower shell than Dash, which is why Debian and friends don't use it as /bin/sh. .ONESHELL mitigates the speed problem, but you could also just use the default shell and leave ONESHELL turned off.

    Use bash strict mode
    ....
    .SHELLFLAGS := -eu -o pipefail -c
I wish people would stop cargo-culting the so-called "strict mode".

The -e flag is only useful because the author likes .ONESHELL mode. If you leave ONESHELL turned off, then you don't need it.

The -u flag is useful sometimes, depending on coding style. I use it on complex scripts. Individual Makefile recipes maybe don't want that much complexity though. Also the -u flag makes the shell's variable-handling behavior inconsistent with Make's.

The pipefail option is Bash-specific, and only works because the author likes to set SHELL to Bash in their Makefiles. It's also not a good default in my opinion. There are times when it's useful, and other times when it's the opposite of what you want. Just depends on the pipeline that you're writing.

[+] jart|3 years ago|reply
The fastest shell is to not use shell special characters. For example, if you say `foo bar >/dev/null` then Make needs to launch your program as `sh -c 'foo bar >/dev/null`. But if you say just `foo bar` then Make can pass that directly to execve(), bypassing the shell entirely. Sometimes I actually do this:

    SHELL := /bin/false
Just to make sure my Makefile doesn't use shell syntax. If you want a `.STRICT` mode, then try Landlock Make.
[+] jwilk|3 years ago|reply
> The -e flag is only useful because the author likes .ONESHELL mode

Not really. The most common class of bugs I see in makefiles is something like this:

  for x in foo bar baz; do frobnicate $x; done
This ignores errors from frobnicate, unless you set -e.
[+] chriswarbo|3 years ago|reply
> Bash is a much slower shell than Dash, which is why Debian and friends don't use it as /bin/sh

Their advice is mostly about using a known interpreter, rather than 'vague POSIX hand-waving'; e.g. they end that section with:

> The key message here, of course, is to choose a specific shell. If you’d rather use ZSH, or Python or Node for that matter, set it to that. Pick a specific language so you can stop targeting a lowest common denominator

Your suggestion of Dash is compatible with that (as is any other particular shell interpreter).

An analogy would be running Selenium tests with particular browsers, rather than using /usr/bin/www-browser and trying to accomodate lynx, dillo, netsurf, ...

[+] yjftsjthsd-h|3 years ago|reply
Title: "Your Makefiles are wrong"

Content: A lot of subjective preferences, with the only thing people are probably doing "wrong" being not properly mapping files as inputs and outputs (which could be a correctness problem but is probably either a mere inefficiency or complete non-issue).

If this had been titled, say, "An opinionated approach to writing Makefiles", or perhaps "How to use GNU Make in a completely unorthodox way that I really like", I wouldn't mind it so much.

[+] ec965|3 years ago|reply
The purpose of a title is both to summarize content and grab the readers attention. It's up the author which one they put for emphasis on. You clicked so it worked, even if you don't like it.
[+] jchw|3 years ago|reply
This seems like a lot of work to not just consider ninja, meson, CMake, etc. I fully understand that the simplicity and portability of Make is alluring, but if you are actually using it to build C software it is a catastrophically poor choice and you can spend a ton of time and effort trying to come close to what you can get out of the box on a modern build system.

If the tradeoff was that Make was easier to use and debug, then maybe it could be justified, but in general my experience is that it's worse.

There are probably some use cases for Make where it remains difficult to replace for one reason or another, but most people using it anymore are not in that position. Now, it's usually more work to keep using it.

[+] hedora|3 years ago|reply
This is a sadly common corrolarry to "those who do not understand make are destined to reimplement it poorly".

I strongly suggest spending an afternoon with "recursive make considered harmful" and the gnu make manual.

I've never encountered a cmake proponent that can add trivial functionality to a cmake build in less time than it took me to learn make.

I can usually port cmake builds to make in less time than such people can debug the cmake version of the build I ported.

[+] gpderetta|3 years ago|reply
I tollerate CMake because a bad standard is still better than no standard, but I would chose Make everyday if it was my decision and just for me.
[+] WesolyKubeczek|3 years ago|reply
Make can do more than compile a bunch of C files. If you can express your goals and dependencies as files with meaningful creation timestamps, it can be a potent task executor that can also skip over steps if they are already done.

Also, your CMake generates Makefiles, so...

CMake and meson/ninja, though, seem to be pretty much tuned to compiling C-shaped things, although I’d like to see them (ab)used for other things.

[+] bin_bash|3 years ago|reply
I thought most people use ninja through CMake? Do people actually write ninja scripts directly?
[+] coliveira|3 years ago|reply
This is bad advice. Changing the default prefix for recipes is worse than using tabs, whatever your feelings about tabs are. Just use make as it was designed, it will work better this way.
[+] blueflow|3 years ago|reply
This results in a pretty un-portable Makefile. Portability is a desirable feature for a build system, which is supposed to help other people on other systems to build your software.
[+] yjftsjthsd-h|3 years ago|reply
Given the explicit choice to use GNUisms and promptly overriding the SHELL to explicitly use bash, I'm quite confident that this author is not concerned with portability concerns.
[+] brabel|3 years ago|reply
What do you mean by portability? Are you able to confidently write a Makefile that works on any Linux distro, BSDs, MacOS and Windows?

Or for you, portability means Linux systems only?

Honestly curious, because I've gone to the trouble of writing my own build system just so I can use the same build file on any OS whatsoever (which to me, means using a cross platform language for everything, not relying on bash or any other shell).

[+] benreesman|3 years ago|reply
This is a really weird thread.

It’s an opinionated blog post about relatively minor Makefile conventions with a clickbait title, which doesn’t look obviously self-submitted.

And yet the thread is #1 and has 96 comments, of which like 92 are trashing the poor guy.

Surely there is someone more worthy of an HN gang-tackle than this? It can’t be that slow of a news day.

[+] Joker_vD|3 years ago|reply
You see, the very first point of the article not only has a really weird opinion (that'd be only half bad), but but this weird opinion is being justified/defended with obviously faulty reasoning: "in the shell spaces matter, [so don't use tabs]. Instead, ask make to use > as the block character". Yeah, because ">" doesn't matter in shell, and there is absolutely no confusion possible as a result. Imagine copypasting a recipe from such a Makefile into a shell? The results will be pretty hilarious.

Which is a shame because the rest of the post is mostly reasonable: turning recipes from a collection of one-liners into an actual piece of shell script, deleting output files on build errors, using -e and -o pipefail, etc.

[+] pessimizer|3 years ago|reply
It's triggering for some people to be told that they are wrong, even if the person who said it doesn't actually know them specifically, wasn't thinking about them as individuals when they wrote the blog, and has no idea that they exist.

The idea that someone could be so presumptuous to assume that they had more make knowledge than everyone on the planet makes them very angry, because they are someone on the planet, so their first instinct is to lash out and prove that person wrong. A better instinct would be to humor the idea that the author doesn't think they know make better than anyone else in the world and isn't trying to hurt their feelings or their careers, but instead is trying to give people who don't know make as well as the author does a few tips.

edit: basically a gathering of the people who reply to things on the internet that upset them with: "That's just your opinion." No shit, buddy, I wrote it, who else's opinion would it be?

[+] shepherdjerred|3 years ago|reply
> Make has a bunch of cryptic magic variables that refer to things like the targets and prerequisites of rules. I mostly think these should be avoided, because they are hard to read.

> However, for the sentinel file pattern, the magic variable $(@D), which refers to the directory the target should go in, and $@, which refers to the target, are common enough that you quickly learn to recognize what they mean:

So, avoid using the magic variables, but actually you should use them because they're useful and common. Got it.

[+] eqvinox|3 years ago|reply
Wow. This is atrocious.

> You really just need the .RECIPEPREFIX = >

Now I can't copy & paste a block anymore (into the shell, to run it), and all my editor indentation settings are broken.

> SHELL := bash

And the Makefile is now non-portable.

> .SHELLFLAGS := -eu -o pipefail -c

If this matters, it's likely you're wedging too complicated things into one recipe. But less bad than the other suggestions.

> .ONESHELL

Funnily enough using this is the primary reason the previous item becomes important. The subtly changed behavior also turns multi-line recipes into a giant footgun if you end up with a non-GNU make.

(skipping a few that are not as bad)

> out/image-id: $(shell find src -type f)

Might be OK in a single rule. Otherwise, it's calling find more... and more...

> Sentinel files

Actual good practice to end it on.

[+] jen20|3 years ago|reply
> And the Makefile is now non-portable.

The article calls out GNU Make, so almost everything else in there is also non-portable.

[+] dllthomas|3 years ago|reply
> Now I can't copy & paste a block anymore (into the shell, to run it)

You likely can, actually. Most terminal emulators have a key you can hold (ctrl in gnome-terminal, alt in urxvt) to select a block of text that doesn't start at the beginning of the line. Doesn't work if your lines wrap, of course.

[+] chrismorgan|3 years ago|reply
> Make leans heavily on the shell, and in the shell spaces matter. Hence, the default behavior in Make of using tabs forces you to mix tabs and spaces, and that leads to readability issues.

I have written a great many makefiles, simple and complex. I can’t recall a single time I’ve needed to mix tabs and spaces in one (though I have had to mix them multiple times in both YAML and HTML).

(As for anything like accidental mixing, for my part I have a sanely-configured text editor and so don’t need to worry about anything silly like tabs being turned into spaces. Tabs are superior to spaces anyway. ⸺But I do use spaces for Rust and Python where that is customary, I’m not completely antisocial.)

> .ONESHELL ensures each Make recipe is ran as one single shell session, rather than one new shell per line. This both - in my opinion - is more intuitive, and it lets you do things like loops, variable assignments and so on in bash.

.ONESHELL also means that your makefile will behave differently from how anyone that’s familiar with makefiles will expect it to. But I guess this does explain why you went enabling strict mode, since you’ve basically turned off the near-equivalent default functionality from Make.

Note also that you can do loops and such already—you just need to use line continuations (put backslashes at the end of each line, which Make will consume).

Yeah, the default behaviour is idiosyncratic and will lead to surprises in the unwary (though they’ll normally observe it immediately, when the cd is ineffective on the next line, or when the if/for causes a syntax error), but I think Make has generally become niche enough that I’d prefer to pander to people that know Make than normal people. :-)

> .DELETE_ON_ERROR

Two-edged sword: it also means you can’t inspect what went wrong by looking at the file. You’re also making the very dubious assumption that merely deleting this one file will fix everything. A few times when I’ve known something to be fallible but want to be able to inspect what it created, I’ve put in something like a `… || { touch --date=@0 $@; exit 1; }` suffix so it still fails, but first zeroes its mtime so that subsequent runs will see that it’s out of date, though the file still exists.

I’m not saying it’s wrong or a bad idea, just that it’s worth considering the implications fully rather than blindly applying it.

[+] jp57|3 years ago|reply
The HN and the blogosphere generally are replete with unconvincing "you're doing it wrong" posts. This is one. I use make (and have used it off and on for a long long time: since the late 80s). I don't do any of these things. If the author is going to make a convincing case his way is "right" and other ways are "wrong", it's incumbent upon him to clearly state what failures I will avoid. Then I can evaluate how often I encounter them, and decide how important this advice is. As stated, it doesn't seem very important.

I frequently run into similar situations with more junior engineers at work. One will insist on dogmatically adopting some "best" practice advocated somewhere, and when I ask what failures or bad situations we'll avoid, or what good situations we'll encourage, they can't answer. In their minds, someone (outside our team or the company) said it's better and so it must be.

I think it's important to avoid invoking incantations, and to understand the reasons for each choice you make. In this article, I don't see that.

[+] dataflow|3 years ago|reply
I don't understand your complaints.

> it's incumbent upon him to clearly state what failures I will avoid

He does exactly that though? Here's a list of some of them:

Rule: "Use a strict Bash mode"

Failure(s) avoided: "your build may keep executing even if there was a failure in one of the targets."

Rule: .ONESHELL

Failure(s) avoided: assignments failing to take effect on subsequent lines ("it lets you do things like loops, variable assignments and so on in bash")

Rule: .DELETE_ON_ERROR

Failure(s) avoided: "ensures the next time you run Make, it’ll properly re-run the failed rule, and guards against broken files"

Rule: MAKEFLAGS += --warn-undefined-variables

Failure(s) avoided: avoids silent misbehavior when a variable doesn't exist ("if you are referring to Make variables that don’t exist, that’s probably wrong and it’s good to get a warning")

[+] Tainnor|3 years ago|reply
I agree that the "you're doing it wrong" tone of the article title is off-putting and that if you're just using Make for some minor automation once in a while here and there, you probably shouldn't worry, but I found most of the tips genuinely helpful and the reasons for doing so are stated or obvious.
[+] avg_dev|3 years ago|reply
I feel the same way. I strongly believe that choice of language makes a huge difference in your material’s reception. Any time I am told that something I’m doing is wrong by an article, an inanimate piece of text that clearly has no cognition thus no idea what I’m doing or not doing, I think that the person who wrote it, is, in fact, communicating wrong.

I was on the fence about posting this reply; after all the guidelines tell us to stay relevant to the material, but I do believe you have done that.

[+] _0w8t|3 years ago|reply
The article does state the reasons for the rules and gives examples how not following them may lead to troubles.
[+] dwheeler|3 years ago|reply
> If the author is going to make a convincing case his way is "right" and other ways are "wrong", it's incumbent upon him to clearly state what failures I will avoid.

I agree, any claim that you should do XYZ should give a strong argument.

The article here does try to give very short arguments, to be fair. I leave unconvinced by many of them. For example, requiring bash means you can't use dash; dash is less capable but much faster.

I prefer arguments that walk through the key pros and cons. Longer, but in long run more useful.

[+] mooselaker|3 years ago|reply
It literally says at multiple points throughout "this is not dogma", including the entire final section.
[+] krinchan|3 years ago|reply
And I think it’s important to read the article before writing comments based purely on the title, but here we are.

The author makes several convincing arguments and specifically lays out what failure modes are avoided for each recommendation. Honestly the title is completely out of step with the tone of the argument, which is a critique I can support.

[+] kazinator|3 years ago|reply
There is a standard way to disable the builtin rules which is

  .SUFFIXES:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/m...

.SUFFIXES

Prerequisites of .SUFFIXES shall be appended to the list of known suffixes and are used in conjunction with the inference rules (see Inference Rules). If .SUFFIXES does not have any prerequisites, the list of known suffixes shall be cleared.

[+] WesolyKubeczek|3 years ago|reply
I think that this is one of actually nicer articles about using Make (and I think that developers should use it in more roles than a glorified task runner. It can do more, look at buildroot!), but the pontificating headline is quite off putting.

I know that Twitter and Medium popularized this style a lot. I wish we used it less.

The tips in the article are interesting, and the text is humbler than one would expect from a preamble like this, though. Go read it, give it a thought.

[+] rcarmo|3 years ago|reply
The RECIPEPREFIX was my first clue that I should not go on reading. And yet, I did, and lo and behold, came to the comments page here to witness most people agreed.

Seriously now I love Makefiles and use them extensively (nothing like make serve and make deploy to simplify my day), but there is a limit where being too opinionated (rather than just simple, easy to understand conventions) just ruins the tool and adds too much cognitive overhead.

[+] cpuguy83|3 years ago|reply
How about "your docker builds are wrong", too.

Don't generate some random id and a tag (as in the post). Use docker's "-iidfile" flag when building to write the actual id of the image to a file which can then be used in a "docker run".

Likewise, you can use "--cidfile" in a "docker run" to output the id to a file and use that later for accessing it.

[+] bin_bash|3 years ago|reply
Can you explain this a bit further? I don't think I understand the point, but I've been using docker more lately and this sounds like it could possibly be something I could use. It sounds like I could do something like:

    docker build --iidfile .dockerid && \
    docker run -it /bin/bash --cidfile .dockerid
Without needing to copy and paste the image ID? Is that right? Why wouldn't I just tag the image?

    docker build -t myimage && \
    docker run -it myimage /bin/bash
[+] throwaway787544|3 years ago|reply
Please don't tell people "you are wrong / don't do X", especially if it's not objectively true. It's negative, rude, judgemental, and bossy. Find a kinder way to make your point and people might listen to you. (If you don't care if people listen to you, why are you speaking?)