top | item 37925929

A Generation Lost in the Bazaar (2012)

95 points| pjmlp | 2 years ago |queue.acm.org | reply

59 comments

order
[+] MrDresden|2 years ago|reply
While the architecture, quality and general decisions made in many (even majority) of FOSS projects can be called into question, this piece makes it sound like this would not happen in projects with 'professional' developers.

The horror shows I have witnessed and had to work with inside some of the companies that have employeed me over the last 15 years have been much worse than anything I have come accross in reasonably large and active FOSS projects.

The Peter Principle lives on strong in both camps.

edits: spelling

[+] api|2 years ago|reply
While this is true, it also remains true that open source is largely incapable of producing anything non-technical people want to use. The very few exceptions are straight clones of commercial products.

I think this failure is more what the article is getting at. What’s under the hood may be shit in many commercial products, in the end they ship something people can use. In the end most users don’t care that the code looks ugly or that the design is messy and full of suboptimal choices. They care that it works and more importantly is usable for them.

[+] bawolff|2 years ago|reply
I feel like its a bit weird all the examples are coordination between separate (possibly competing) projects, when the implied solution is having a single person responsible for quality. That doesn't make sense to me.

Sure you could have a single person responsible for quality within a project, and that may even be a good idea (BDFL is a pattern for a reason). However you can't do that across organizational boundries. Ports will have lots of duplication because the entire point is to bring the wider world together including projects competing against each other.

Like the solution just doesn't seem applicable here. It feels like the equivalent of someone suggesting to fix the problem of firefox & chrome doing the same thing by appointing a single person responsible for web quality. The solution doesn't seem wrong so much as non-sensical.

I don't particularly see the connection to cathedral & bazaar either. Even before that book competing software projects had duplication.

[+] mananaysiempre|2 years ago|reply
The implicit point of comparison looks to be the monolithic Unix of old, where the entirety of the code of the system is (hopefully, ostensibly, allegedly) developed with a single vision, except for the application programs in the narrowest sense of the word (e.g. the particular piece of numerics you needed to design your airplane or whatnot, but not the editor you used to write it, the cluster scheduler you used to run it, the typesetter you used to write down your results, or the 3D viewer you used to present them).

As far as I understand, that—compared to the Linux distro which derives a similar range of capability from a primordial soup of small nuggets of software—was Raymond’s “cathedral” as well, and the original model of the BSDs, before the range of expected functionality was broad enough that distro-ish “ports” became unavoidable.

One example that’s just plain interesting to read to see what things people were exploring (because, let’s admit it, historical Unix source gets plain boring after a while) is AT&T Research’s https://github.com/att/ast.

[+] RetroTechie|2 years ago|reply
Once you have a certain size codebase / ecosystem, removing unnecessary complexity from it is a much harder skill than adding more.

Some 'law' about always something, somewhere depending on whatever got built previously. Documented or not.

Of course there's the rewrite. Read: starting over with a better design than existing one. Yet another skill that few mortals possess when it comes to intricate machinery like an OS.

And then a whole bunch of functionality must be re-implemented. Which takes countless human-hours (always in short supply). If not years.

That leaves the option to replace pieces one by one with better ones, leaving overall structure (mostly) in place / working. Oh wait, that's what is usually done. Rip something out, plug the hole, move on.

[+] spacecadet|2 years ago|reply
I worked with an engineer once who spent most of their time refactoring and cleaning up. They explained over beers how they wanted to spend more time being net-negative. After they left, sure enough, removed more code than ever added and was the best member of the team. That stuck with me and I also now strive, where it makes sense, to remove before create.
[+] giovannibonetti|2 years ago|reply
> Well, autoconf, for reasons that have never made sense, was written in the obscure M4 macro language

This immediately reminded me of Helm charts (Kubernetes) and their implementation with Go templates, that work in the syntatic level instead of semantic, which makes it unnecessarily hard to operate it.

[+] topspin|2 years ago|reply
Helm templates literally make me nauseous. I don't mean literally in the smart-alecky, Zoomer sense. I mean actual nausea.

It disturbs me that Helm templates became a defacto standard for k8s. Something went very wrong there.

[+] Animats|2 years ago|reply
And get off my lawn.

Few forces favor convergence and standardization in open source. After a few decades of this, we have far too much duplicative stuff.

It's really hard to clean that up. At one time I tried to get the 6 or 8 Python packages that parse ISO8601 date strings standardized. All of them had bugs. It took 6 years of bikeshedding discussions.[1] Issue filed in 2012, patch applied in 2018. For something that's a few screens of code.

[1] https://bugs.python.org/issue15873

[+] AdamN|2 years ago|reply
Over time I've concluded that a single standard for anything important simply isn't enough. It's best to accept that 2 standards and optionally a 3rd and 4th slot for upcoming standards are the best that can be done. Fixating on getting to one is actually counter-productive.

This isn't just true of standards, it's also true of implementations of a standard.

[+] hashar|2 years ago|reply
> datetime.fromisoformat(date_string) > > Return a date corresponding to a date_string given in any valid ISO 8601 format, except ordinal dates (e.g. YYYY-DDD)

Thank you for that!

[+] eviks|2 years ago|reply
Yeah, the last mile is the hardest
[+] antonvs|2 years ago|reply
I love this article for its critical perspective. We need more of that if we want to move past the rather unimpressive status quo.

But, the reason for the described situation is no mystery, and may not have that much to do with the bazaar. Programmers like to reinvent stuff, as well as use what they’re familiar with. Those two facts alone explain a lot of the criticisms.

I suppose, in a properly managed cathedral, these tendencies might be somewhat mitigated. But how many cathedrals are properly managed, or have the luxury to “do things right”?

The whole “worse is better” debate is relevant here. Worse may not be better because it’s actually better, but because it’s more easily achievable, particularly by people without infinite time, expertise, and money.

[+] harimau777|2 years ago|reply
I'd add onto this that it is in the programmer's best interest to reinvent stuff in order to improve their technical skills.

It would be interesting to see a company provide professional development so that programmers feel less pressure to do so in their day to day jobs.

[+] vba616|2 years ago|reply
A thing that seems incredibly rare these days, in my perceptual bubble anyway, is an online address form that lets you enter US states on the keyboard with the standard postal abbreviation.
[+] bruce511|2 years ago|reply
I could not have articulated it like this, but looking back on my career i can see it playing out, in my own work and the work of others.

But it's probably worth pointing out that while cathedrals are built, with one architect's vision, not all people are capable of envisioning a cathedral. Many see only ugly 4-walled-flat-roof sprawl. They might build it really well, but its still sprawl.

Equally, things improve with practice and time. My early work may be "single vision", but I'm more skilled now, my vision is clearer and my execution of that vision is improved.

Wonderful software, like wonderful buildings, are wonderful partly because it is not common.

Good vision is partly driven by talent, partly by experience - good software is the result of both great architecture and a capable team willing to work in service of that vision.

Of course all good builders want to be architects, and few want to be remembered as the successor to someone else's vision.

[+] lmm|2 years ago|reply
Meh, this does the rounds every so often, but the fact remains that auto* work a lot better than everything that's supposed to replace them, especially if your system is even slightly obscure. Want to build a unix program under SUA? If it uses autoconf, you're fine, just push the button and it works; at worst if the release is older than your OS then you may have to replace two files, a standard and well-known procedure. Want to build varnish under SUA? Lol, good luck.

The problem of duplicated dependencies in linux comes from doing too little splitting of packages, not too much. You get a lot less of this on a platform like nodejs where people properly split up functions into much smaller libraries, because the dependency management is much more effective. (Of course people on HN moan more when you do that, for no discernible reason, as if having 1000 dependencies is somehow morally inferior to having 10).

Ultimately a certain amount of duplication is necessary and healthy. There is a nugget of real criticism to be found here - the "Bazaar" nature of open-source OSes makes it very difficult to deprecate and remove anything, where a more centrally-managed system is able to make decisions like "we'll stop using or supporting Perl from 2020" and enforce them. But "herp derp 122 packages herp derp 22 tests" is not what we should be focusing on.

[+] oblio|2 years ago|reply
> Of course people on HN moan more when you do that, for no discernible reason, as if having 1000 dependencies is somehow morally inferior to having 10

It's not morally wrong, it's wrong for auditing and general change management. Too many suppliers of small dependencies you have to check.

This is important for larger companies or for mission critical software in sensitive domains.

[+] cpach|2 years ago|reply
As someone who is an outsider to Node/Javascript, I’m not convinced that NPM has found the sweet spot of package granularity…
[+] oblio|2 years ago|reply
SUA?
[+] ggm|2 years ago|reply
Hmm. He might rail against /usr/ports/ but it was better than the alternatives of the time (I note pkgsrc, maybe comparable, certainly similar)

His beef is that people are lazy. If there is a perl tool to munge something you use it, even if the rest of the process uses Python. This is the reductionism of "if you start in awk stay in awk" which is good, honest and .. hard.

Ports is better than APT. apt, has the disease of 'I made a bundle for you but the name is not indicative of whats in it' -where Ports has metaports which are explicit what they pull in to make the "thing" they meta over.

APT has the "-dev" norm now. thats ok, but sort of stupid too. How can it possibly make sense to have to pull in the -dev thing to use its shared libs standalone in another package? (I have seen this)

Ports is the bazaar but with sub-zones. Like the Grand Bazaar in Istanbul, things tend to clump together. Sometimes you wonder why its in sysadmin/ and not text/ but in the end its birds-of-a-feather.

Homebrew ditched M4 for Ruby. Gak.

CPAN is the enemy.

[+] bgribble|2 years ago|reply
The yearning in this essay for an era of wise craftsman-philosophers building architectural monuments for the ages is just fantasy. It never was and never will be.

I remember great pieces of software from before the Internet era. The grammatical tense that applies to them is "past". They are dead and gone and only exist as "hey, remember Code Warrior?" "remember the LISP Machine?" "remember Lotus Notes?" Millions and millions of dollars, hours, lines of code all just gone. They live on as inspirations but not as artifacts.

The "cathedral" model doesn't produce cathedrals, it produces photos of cathedrals that were abandoned and eventually collapsed and were forgotten.

The bazaar has produced code that outlives its authors, its sponsors, its original communities. You can Tanenbaum Linux all day long but it will outlive the cockroaches. That's the real monument.

[+] cpach|2 years ago|reply
I’m not really sure what one is supposed to take away from this rant. (It’s also quite old so some things have changed in the years that followed.)

In the article, PHK seem to assume that it’s a good idea that each user compile all their applications locally. I know this is the norm in some places (e.g. Gentoo?), but is it really a reasonable expection of users in general? I consider myself a power user, but if I want to install a tool, in most cases I just fetch it using Homebrew and I will get a binary, so no need to compile it at all. Compilation and packaging might be complex for some of those tools, but it’s “out-sorced”. IMHO a system such as macOS + Homebrew works well from the end-user perspective.

Do I say that just because I’m lost in the bazaar? Have I even seen one of these cathedrals that PHK refers to? Not sure. What would count as a cathedral? HP-UX? Symbolics’ Genera? Would OpenBSD count?

[+] fergie|2 years ago|reply
"there is no escaping that the entire dot-com era was a disaster for IT/CS in general and for software quality and Unix in particular."

Strongly disagree. We need to get back to the utopian ideals of the early web where normal people were encouraged to cobble together weird and wonderful websites.

[+] anthk|2 years ago|reply
Also, Linux 2.4 and FreeBSD 4.x were rock solid, almost at the end of that era.
[+] cpach|2 years ago|reply
But isn’t that example more about publishing and not software development?
[+] DonHopkins|2 years ago|reply
“My favorite part of the “many eyes” argument is how few bugs were found by the two eyes of Eric (the originator of the statement). All the many eyes are apparently attached to a lot of hands that type lots of words about many eyes, and never actually audit code.” -Theo De Raadt
[+] treyd|2 years ago|reply
Has anyone tried developing a project to machine translate all these autoconf/etc configurations to something more modern like Meson that's been able to learn from past mistakes? Like obviously there's many hacks over the many years that wouldn't be digestible and able to be translated but surely it can be done on a best-effort basis.
[+] lmm|2 years ago|reply
Yes, these things have been done, but largely end up being abandoned. If the maintainer doesn't want to port to Meson or whatever your alternative is, they're pretty useless - your "best effort" port will break arbitrarily. If the maintainer does want to port to Meson, it's probably easier for them to write what the build should be from scratch than pull anything out of autoconf.
[+] anymouse123456|2 years ago|reply
Extra! Extra! Read all about it! Old man shakes fist at clouds!

Seriously though, the problem is that the Cathedral is a Utopian fantasy that has never once materialized in our timeline and the people who attempt to instantiate it invariably create something much, much worse than the Bazaar.

It may look like it's made of stone, but those arches are made of painted poo.

Sure, the Bazaar is ugly and even offensive to the refined eye of the great priesthood, but these things are almost always far more useful because they come from necessity, closer to the person with real needs.

[+] mvac|2 years ago|reply
> ... quality happens only if somebody has the responsibility for it, and that "somebody" can be no more than one single person

So true, I've yet to see anything of quality produced by groupthink.

[+] bitwize|2 years ago|reply
This is a poorly-thought-out screed by longtime BSD hacker Poul-Henning Kamp, a.k.a., someone who should freaking know better.

Autoconf doesn't exist because "the bazaar" is full of lazy incompetents and only cathedrals have halfway decent design. It comes from the bad old days when "open" meant "spec documents are available under RAND terms" and had little to do with what we now call open source. Unix had splintered into lots of different idiosyncratic systems, each with its own configuration, and the GNU stuff had to run on all of them. Autoconf provided a portable way to paper over all those differences.

The same with libtool. The Unix splintering had already happened when shared libs became relevant for Unix. There wasn't even agreement among the various vendors on how to implement shared libs! For example, how do you resolve the fact that a dynamic/shared library could be loaded anywhere into a process's address space, making absolute pointers inside the library invalid? There were two main schools of thought:

* when linking the library, replace all exported pointers with placeholder values. Then when the dynamic linker loads the library, it swizzles the placeholder values back into pointers. (Windows)

* use position-independent code, taking advantage of relative jumps, calls, and data accesses to create an object file that can be loaded anywhere. (ELF)

I've seen some half-measures as well:

* give each library a fixed address where it's loaded every time (Linux a.out)

* punt on symbol resolution; hand the programmer a base address and have them call into the library at fixed offsets from the base (AmigaOS through 3.x)

Each of the implementation strategies for shared libs required a different constellation of compiler and linker flags. Again, libtool papered over these and made it tractable to write shared libs for a variety of operating systems. It is true that these days, compiling with 'gcc -fPIC' and linking with 'gcc -shared' will probably do what you want. But that assumes a modern enough OS and toolchain, something that GNU couldn't and still cannot in some cases necessarily assume.

The purpose of both these programs is to portably solve problems that were created by multiple, competing cathedrals. I don't like Autoconf or Libtool. They're unholy messes. But they're largely that way out of necessity considering the problems they solve, problems which would otherwise be nigh-intractable -- not because their developers were lazy or incompetent.

And no, Meson does not solve the same problem, it punts on working on any OS that's insufficiently modern.

The neat thing about the open source bazaar is that it can produce reference implementations for things everybody agrees should be in the finished product, reducing the friction of competing implementations. An example is the 86open consortium, an effort by various proprietary x86 Unix vendors (mainly BSDi, Sun, and SCO) to agree on a single binary standard for x86 Unixes. Well, the vendors got together fully expecting to get into the weeds with meetings and committees and stuff... but what they found was all their products had Linux/ELF kernel personalities already built in. So they declared Linux/ELF to be the binary standard and disbanded the 86open initiative!

[+] Pinus|2 years ago|reply
Autotools was a perfectly reasonable way of solving 1993 problems with 1993 tools. Whether it still makes any sense in 2023 a different matter... Having said that, given the choice between a non-working autoconf build scheme, and a non-working cmake build scheme, I prefer the former, but that may just be me getting old and lazy. Obviously, a working setup of anything would be much preferrable, but that seems like hopeless case if one is using anything other than the most popular Linux distro at the moment...
[+] cmrdporcupine|2 years ago|reply
Exactly this. While I'm not a huge fan of the "Bazaar" as a development model, I'm even more terrified of the "Cathedral" and/or anybody who has an idea that there's one-true-way to do things (and it's always theirs and only theirs).

Behind that scary door is the war of architects and their egos, splintering half-completed standards, and nightmares from the past like CORBA etc -- maybe-decent on paper, but good luck ever finding an open and usable and actual quality, working implementation.

What we need is to develop consensus on how to do things, and that, unfortunately, is messy, takes time, and involves people compromising all over the place. And I'm not convinced that committees are the right instrument for that either.

The essay's slagging of the .com era is funny to me as well; because I lived through that era and was absolutely one of the messy green hackers/devs that he is complaining about. Funny thing is though that things shipped, still, and it led to a lot of people being able to make web pages, write code, and get themselves out on the Internet who hadn't had that privilege before.

And it wasn't the quality of the software that caused the .com implosion after, it was the quality of the business plans.

And I also saw what came after, the kind of reaction to the .com chaos; it was all J2EE and EJBs and big enterprise application servers, and I never worked in a single shop where that actually led to anything quality and saw more than one where a team that was doing Perl+CGI+Apache ate itself by rewriting everything "the right way." A few years later, the trend swung back the other way; Ruby on Rails, web2.0 etc. etc.

Software development is messy. That's just how it is.

[+] bvan|2 years ago|reply
Very good essay. The new cathedrals are being build on very loose foundations these days.
[+] cpach|2 years ago|reply
Could be. Do you have any specific examples in mind?