What's in a Build Tool?

[xyzzy_plugh]

Regarding make, the author states:

  Parallelize different commands: Okay: There's no automatic way to parallelize different targets, though you can still parallelize a single target using Bash.

This is wrong, make has supported the -j flag since 1988:

       -j [jobs], --jobs[=jobs]
             Specifies the number of jobs (commands) to run simultaneously.  If there is more than one -j option, the last one is effective.  If the -j option is given without an argument, make will not limit the number of jobs that can  run
             simultaneously.

[microcolonel]

I can't believe anyone could use make in the last ~12 years without the -j flag. We've had dual cores at least for ages.

[herge]

It's a pity that rake is not included in the table at the bottom.

One huge advantage of rake is the ability to easily debug what is happening. A quick `p [filename, task]` lets you know what is happening. Even, if needed, you can quickly do

    $ irb
    >>> require 'rake'
    >>> p FileList["foo/*"]

and actually play around with the libraries. If you know anything about ruby, and especially if you are looking to build anything other than a c/c++ project, it is definitely worth looking at.

[luckydude]

I thought this was going to be a rant against autoconf. When that rant comes I'll post our 186 line shell script that does what autoconf does for pretty much any platform. Handles AIX, Solaris, FreeBSD, HP-UX, IRIX, Linux, NetBSD, OpenBSD, SCO, SunOS, Windows, MacOS.

I hate autoconf but that's not part of this thread. I'll wait.

[bhaak]

Unix, Unix, Unix, Unix, Unix, Unix clone, Unix, Unix, Unix, Unix, Windows, Unix.

So, your shell script handles Unix, its most popular clone and Windows. ;-)

The Windows part is interesting. What are the requirements to run your shell script on Windows?

[Ace17]

I'm curious about this shell script ; this looks to good to be true!

Having automated the build of dozens of FOSS packages, I can definitely say that autoconf-based packages are _by far_ the easiest ones to build ; especially if you want to do funny stuff like out-of-tree builds or cross-compilation (can your 186-line script do this?).

Some projects (x264, ffmpeg) provide 'configure'-like custom scripts, trying to imitate what autoconf does ; but in practise you can't automate their build the same way you did with autoconf-based projects.

[qznc]

Why not write a preemptive blog post?

[yeowMeng]

Can you please GNU me yr 186 lines of bytes. I am curious.

[codyps]

ninja? tup?

Article doesn't note the property of being able to depend upon the entire command that generates an output (ie: re-generate when compiler flags change). This is something that makes doing reliable builds much easier (when present). It's notably very hard to do in make (and even then is very inefficient).

Also, on "download" the author seems to presume that one takes the naive approach in each tool. In most cases, if one spends a bunch of time on it the downloads can be done fairly efficiently (especially in make, without even much work there). Most of these build systems are fully programmable, so the rating should probably focus more on the level of difficulty to do well (with some requirements specifying what "well" is)

[thelema314]

To depend on compiler flags, I do this:

## compiler_flags file keeps a copy of the compile flags $(builddir)/compiler_flags: force mkdir -p $(builddir) echo '$(CPPFLAGS) $(CFLAGS)' | cmp -s - $@ || echo '$(CPPFLAGS) $(CFLAGS)' > $@

##rebuild all objects if compile flags change $(LIBOBJECTS) $(RTLLIBOBJECTS) $(OPTLIBOBJECTS) $(TESTOBJECTS) $(builddir)/init_qt_workdir: $(builddir)/compiler_flags

I"m pretty happy with the results.

[Debilski]

Depending on compiler flags was implicitly mentioned for sbt, though (point 8). Depending on the `.value` of the `scalaVersion` configuration property is the same thing and can be done for all other settings input as well.

[falsedan]

I think the author does a great disservice lumping together deploy, test, and dev sandbox tools under 'build tools'. They are different scenarios which all happen to require a built copy of the code.

There's no fundamental reason for them to use the same tool, except the inertial tug of whatever build tool your project happens to use.

[djKianoosh]

Gradle is a glaring omission if maven, ant and sbt are included.

[jschwartzi]

I'm curious what the author thinks of CMake. It reminded me a lot of Maven, but for C/C++ projects.

[codyps]

The author also doesn't cover autoconf/automake, which operates in the same vein as cmake (generating files for use by another build tool).

I agree, it would be useful to evaluate it here as it provides a bunch of the features the other "build tools" provide.

[imron]

For c and c++, I've recently starting using meson.

It's a breath of fresh air compared to some other build tools, and the config files are both expressive and concise.

[Too]

Am i the only one thinking that doing a simple compile is something that should be handled fully by the compiler and not the build tool. The compiler should automatically detect which files are being #included and allow for incremental compile based on this. The Typescript and SASS compilers are good examples of this, you just invoke it like 'tsc main.ts --watch' and it will automatically know that main.ts included other files and these will also be watched.

If your compiler does not support this, the only way to build a proper dependency tree is 1. if the compiler can generate one in a standard format (gcc -MMd), 2. your buildtool knows as much about the language as the compiler does, 3. you write the dependency tree manually.

1. Is bad because it requires the compiler to support generating a wide range of dependency-graph formats, or every build tool must support the make syntax.

2. Is bad because the build-tool becomes one massive monolith and there is no way they can reliably support every language in the world.

3. Is bad because humans are lazy and make mistakes.

For the build tool, this leaves it's responsibility to doing things before and after compilation (generating/preprocessing code, copy assests, run tests, deploy to staging), which usually means invoking arbiatry and custom shell-commands, the order of this is always so application-specific that #3 is inevitable. If doing these things are complicated, your build tool sucks, and sadly i found most build tools to fall into this category because they try #2 and want to be the center of the universe. Even doing a simple thing such as copying a file requires you to learn a new awkard xml-syntax, install the "copy-file-plugin" or write your own copy-file-plugin in the awkward syntax. What i would want is somthing with the simplicity of Gulp or Make, but with a bit more widespread syntax and more batteries included for file-manipulation, like Python would be perfect. Rake looks promising, maybe it makes it worth learning Ruby?

[lmm]

Disagree. Compilers, like anything else, should be small composable libraries. Compilation and dependency parsing should perhaps be separate, and the same dependency-tracking library should be usable by the compiler, wider build tools, and IDEs or similar.

[Ace17]

If the syntax is the only issue, maybe we should fork GNU make.

[lasermike026]

I was just writing a Rakefile and have a good time of it. I like using make but rake gives me more functionality.

[shoover]

Rake is a pleasure for small projects, like generating HTML from a handful of org-mode files or zipping up a Chrome extension. I've not tried to take it much farther than that, though there is a series of blog posts I can't find attached the moment in which the author used rake for real C projects with GCC.

[gkya]

The OP mistakes what a build tool is. A build tool generates the executable from sources. Use other tools for other tasks. I once wrote a tool for running commands on file system events, but I lost it. But for example I write files rather often, a build on each write would be annoying for me. It's a personal thing, so give me the build script and let me use it the way i like.

[dlandis]

Ant and Maven are fairly obsolete at this point in the Java world having been superseded by Gradle, and so aren't really worthy of much discussion. But Gradle is quite good and I think it would stack up very well against the others including sbt. Like sbt, I think it fuses a task based and configuration based approach and also does dependency management.

[pkd]

Definitely not Maven. Maven is ubiquitous in Java projects. Gradle has been pushed hard by Android, but I know that a lot of developers aren't convinced that putting code into configuration is a good idea. Maven even after all the XML crud remains insanely flexible and powerful.

[smadge]

Yikes, I still use Maven. I don't like being left behind in the dust. Is there any compelling reason to prefer Gradle? What mistakes did Maven make that gradle avoided? One thing I am hesitant about is the I don't need my build tool to be a turing complete language.

[tdurden]

Ant maybe, but Maven is hardly obsolete. Most every Java project I run into uses Maven or Gradle, with Maven being more common.

[kzhahou]

Gradle has superceded Maven in the same way that Node has replaced Java and C++ in production (hint: it hasn't).

[aikah]

Isn't Gradle based on and using Maven (under the hood) ?

[davidpronk]

After reading this article on NPM as a build tool I immediately gave up on Gulp and Grunt. Highly recommended! http://blog.keithcirkel.co.uk/how-to-use-npm-as-a-build-tool...

[manaskarekar]

A nice summary of some build tools:

https://bitbucket.org/scons/scons/wiki/SconsVsOtherBuildTool...

[shoover]

This is helpful and uses a good matrix to compare the tools in question. I've been wanting to see a version of this for cross-language tools: Bazel et al, msbuild, scons, and do not forget the quiet but powerful waf.