top | item 39375788

Why is Common Lisp not the most popular programming language?

130 points| kryptiskt | 2 years ago |daninus14.github.io | reply

332 comments

order
[+] samatman|2 years ago|reply
It's the lists.

No, not the prefix notation, parenthesis, what have you, although that doesn't help. The lists themselves. In Lisp, code is data, and data is lists.

Yes, of course, there are hashmaps, arrays, strings. But idiomatic Lisp code really does use linked lists extensively, it's an entire style of programming. Even if you'd prefer to use different data structures (and again, Common Lisp does support this), you still have to be able to read and reason about the many parts of the language designed for, as the old quip had it, prothething lithts.

The "weird looking syntax" does not help, but Lisp hackers are right that you get used to that part, might even appreciate it if macros are something you really care about. Structural editing a la paredit and parinfer are pretty nice too.

But when it comes with a weird way of programming, that's a bridge too far for a lot of people. It's harder to learn, read, and reason about, for longer than most languages.

I'm glad they mentioned Julia though. Learned everything Dylan had to teach and then some, real treat of a language, and, it turns out, writing macros without cons cells is rather pleasant and powerful. Certainly an acceptable Lisp in my book. Maybe someday they'll add conditions and restarts, that's the one feature CL which gives it an edge on Julia.

[+] jjav|2 years ago|reply
I didn't see mentioned in the thread (maybe missed it) the #1 reason, IMO, that Lisp can't be popular in companies.

Everyone should take the time to learn Lisp and do a handful of personal projects with it. It'll help your growth as a software engineer. Just do it!

But that doesn't make it a great corporate language. Lips is infinitely flexible, you can mutate it to be what you want. That's cool and feels awesome.

Also: a maintenance nightmare as soon as you have more than ~1 person working on the codebase!

Everyone surely has heard the joke/truism how C++ can be great as long as you only use a sane subset. But every team uses a different subset. Now imagine something like Lisp where every developer & team morphs it in a different way and you have a product with hundreds of developers on it. Try maintaining that without going insane.

The anti-Lisp is something like Go. Simple, not very flexible, everyone does it the same way mostly, you can plug & play developers like scrum demands we do.

[+] Karellen|2 years ago|reply
> But idiomatic Lisp code really does use linked lists extensively,

Idiomatic python code uses lists extensively. In my (somewhat limited) experience, they're the default data structure to use for a lot of algorithms.

Slightly more accurately, a lot of stuff in python uses sequences extensively, which are implemented by a number of types - but the default sequence is a list. And strings are lists. Yeah, if a list doesn't cut it then try sets or tuples or generators or coroutines or something else that implements a sequence. But for your initial prototype? List.

And python's pretty popular.

[+] funcDropShadow|2 years ago|reply
> It's the lists.

That is one of the defining differences between Clojure and older Lisps. Clojure has list, vectors, maps, and sets as equally well supported data structures in the syntax and standard library. Classical Lisps often miss proper abstractions over different data structures. Clojure has those as well.

[+] CyberDildonics|2 years ago|reply
I think it's strange that anyone would even ask why common lisp isn't popular. There is no payoff to writing something in it. The binaries are huge, it is going to be a lot less clear than modern C++, it won't be as fast, you still have a garbage collector, the libraries are niche, the syntax is reversed, the tools are niche, the ecosystem is niche, and everything you do is the opposite of what is intuitive at first.

Then on top of all this is built on linked lists which are essentially an obsolete data structure in their simplest form.

There is no reason to learn something with backwards syntax and ancient tools when there isn't even any payoff. Write something in C and the program is fast, small, native and can be compiled anywhere in a fraction of a second. There is still a payoff for all the very real pain. In lisp there is just no reason to use it from any angle other than how clever someone can be with macros and that is the exact opposite of good sustainable programming.

[+] packetlost|2 years ago|reply
I don't think it's the lists. I think it's the cons cells. Lists are pretty ergonomic in most languages. As someone who loves Lisp (Scheme, specifically), cons cells are elegant for the compiler nerds and a complete nightmare for the programmer. Fuck `car`, `cdr`, `cdar`, `cddar`, `caar`, etc.. It's a horrible interface and every time I need to remember which variant gets me the value stored at the tail of a list I get a little more angry and want to write more Clojure and less Scheme.
[+] IshKebab|2 years ago|reply
I disagree. I have no problem with lists obsession in other languages like OCaml.

The parentheses are just too much. I get how elegant and unambiguous they are for computers but I am not a computer. It's like RPN. It's elegant and easy for code to parse and unambiguous and all these nice things.... except it isn't easy for me to parse.

Compilers are perfectly capable of compiling readable code like Rust so I don't see why I should have to do the tedious work of figuring out all the parentheses manually.

Lisp is like a really great IR. But I don't want to program in an IR.

[+] breather|2 years ago|reply
I'm tackling a significant lisp project right now, and the thing that holds me up right now is that the code is difficult to organize. Python, java, rust, go etc have well-defined patterns to figure out where code lives and where you might expect certain behaviors to occur. With lisp you can really shoot yourself in the foot very easily by using abstractions that are difficult to follow and are spread out across many files.
[+] taeric|2 years ago|reply
I'm trying to understand why lists matter. Yes, there are some oddities where some base constructs don't play as well with hashmaps as they do with lists, but it is far from difficult to work with a hashmap. Or an array. What difficulty are you talking about, specifically? (Genuine question.)

Biggest difference I know of, off the top of my head, is the LOOP macro having constructs that care which one you have. Curious if there is more.

[+] naveen99|2 years ago|reply
More specifically, the linked lists are too inefficient for modern computing to be the default data structure. Not enough locality, too much jumping around memory. Reality is a tree, but to organize it, you have to convert it into arrays, maps, and matrix multiplication.
[+] valty|2 years ago|reply
It's funny that behind the scenes in all languages, everything is lists (in the AST).

It's strange that we tend to initially code in easily readable built-in control structures (if, switch, for, while), but then there always comes a point when you need to refactor...and convert them into lists of data structures and process those.

For example, if you are matching routes in a web server, you can write a bunch of if-statements. Very simple. Easily understandable. And you can use whatever criteria you want, in whatever order you want.

    if (request.pathname == '/foo') { return foo }
    if (request.pathname == '/bar') { return bar }
But say now you want to print a list of all the routes and how they are matched.

Most people create a concept of a `Route` object that contains a predicate, and then you loop over those in a list. It feels super clean. But now if you want an exotic way of matching a particular route, or you want route priority or anything like that, and your route matcher and Route objects start becoming really complex...but if you did it with a simple code block with if statements, it would have been really easy.

    const routes = [ 
      {pathname: '/foo', action: () => {}}, 
      {pathname: '/bar', action: () => {}}, 
    ]
    for (const route in routes) {
      if (route.pathname === request.pathname) { return route.action }
    }
If we could have referenced our if-statement code blocks as a list (using Reflection or something), then we could have avoided any abstraction and stayed totally flexible.

I could easily throw a few more requirements at you, and you would quickly have some frankenstein Route object and matching logic.

It's this weird process of "dont-repeat-yourself" where everything looks the same and you abstract it, and then you realize its not all the same, and instead of back-tracking the data structure, you just tack on new stuff and more complicated logic.

Complexity in software stems from these pre-mature abstractions.

[+] habibur|2 years ago|reply
It's vector and hashmap that you need most, not linked list.
[+] CodeWriter23|2 years ago|reply
+1 for writing “processing lists” with a lisp
[+] linguae|2 years ago|reply
Common Lisp once had plenty of backing from major companies like Symbolics, Xerox, and Texas Instruments, as well as smaller companies such as Harlequin. Even Apple owned Macintosh Common Lisp at one point, and SK8 (which could be considered a follow-up to HyperCard) was implemented in it. Had Apple not had its problems in the 1990s, one potential alternate version of Apple I could imagine would be some sort of Lisp OS with a Mac interface (true story: the Newton could’ve had a Lisp OS if it weren’t for C++ advocates winning out). Unfortunately, the AI winter of the late 1980s and 1990s severely damaged the Lisp market; companies either abandoned Lisp or went under. The torch of Common Lisp has been carried on by open source and commercial implementations, but there are no major companies actively backing Lisp in the way that Java is backed by Sun/Oracle and that Python is core infrastructure at countless companies, large and small.

I didn’t know that conferences could potentially generate a lot of revenue to help fund programming language development. I’m quite intrigued by this idea, actually. Thanks to advocacy of the language, there are probably thousands of Common Lisp developers in the world, many of whom are working on interesting projects. Perhaps a series of profitable Common Lisp conferences will provide the spark needed to come up with a modern Common Lisp foundation that is able to support further development of the language.

[+] gumby|2 years ago|reply
> ...one potential alternate version of Apple I could imagine would be some sort of Lisp OS with a Mac interface...Unfortunately, the AI winter of the late 1980s and 1990s severely damaged the Lisp market...

Your order is messed up: work on Dylan commenced years after that AI Winter had begun.

[+] hajile|2 years ago|reply
The saddest thing to me is that Brendan Eich didn't do a scheme like he intended.

If that had happened, the web would have been fast 15 years earlier. Things like Flash wouldn't have ever happened. ES2015 would have never been necessary. HTML would have gone away. CSS would have never existed. Declarative web frameworks would have become a reality decades ago. WASM wouldn't have happened. Even stuff like the App Store likely wouldn't have happened because Scheme would have been fast enough that Jobs would have stuck with his initial web app idea.

[+] binary132|2 years ago|reply
None of these "programming language evolutionary fitness retrospective" discussions ever manage to address the real heart(s) of the question:

- UNIX is C, so C-family languages won.

- The network effect is by far the most dominant force in the matter.

All of us would like to believe that we operate in a world where technical merit determines memetic fitness, because our jobs seem to depend on it: after all, if we build good things, we immediately see the consequences, and others must also see and appreciate these consequences, right? By doing well at that, we succeed, right?

Well, it turns out it's not so much like that actually.

[+] NikkiA|2 years ago|reply
MacOS, with a slight help from MS, very, very nearly tipped the balance to Pascal in the 80s. I dare say that without C++ hitting mainstream around 1990, C would have languished.
[+] zoogeny|2 years ago|reply
I admit that I haven't touched CL in decades and my most recent exposure to Lisp-ish stuff is writing a bit of Scheme and reading a bit of Clojure (maybe some elisp too).

I like a lot of the ideas behind Lisp-like languages but I dislike writing in them. Parenthesis, as others mention, start to get unwieldy. I often feel like I'm expressing myself "in reverse", almost like I'm programming in Yoda speak. I mean by that, it often feels like I want to edit/add-to the beginning of a statement rather than the ending of it.

Perhaps the discomfort would dissipate with use. But to be honest, I'd rather just grab Python or even Typescript these days for the kind of ease and flexibility where raw performance doesn't matter. And there are other programming languages that excite me more than Lisp (e.g. Elixir, Unison) so my limited free time has plenty of outlets on that front.

What surprises me is that Lisp evangelists have this default assumption that their language ought to be entitled to popularity. They have some kind of "if people only knew" mentality, just like in this article, where they assume that the only reason people don't flock to their language is ignorance. Perhaps it doesn't occur to them that people try it out and then leave and never come back.

[+] derp38726|2 years ago|reply
>Perhaps it doesn't occur to them that people try it out and then leave and never come back.

Once the code is data is code clicks there’s no going back. When you read other languages you just see a lisp DSL, you can’t unsee it because you’ve changed.

[+] thot_experiment|2 years ago|reply
Because most programming is done by people trying to do a job and it requires broad tooling support and familiarity, LISP is a language that has neither.

LISP, (similarly to Haskell) requires you to bend your mind and pay an upfront mental cost in order to access it's benefits, which are that everything is equally easy to describe. No construct in LISP feels like it requires you to bend the language in an awful way because LISP is a fantastically generic tool, some things that are common and easy in other languages are more difficult in LISP, but as you get into the weeds building more complex constructs it will never get in your way.

(and since i mentioned it, the benefit of Haskell is that you can make a LOT of statements about your code you know to be true)

[+] JohnFen|2 years ago|reply
I didn't find Lisp to be particularly costly in terms of mental energy to become competent in. Rust, however, was a heavy lift.

Perhaps different people are inherently more or less compatible with different languages?

[+] PaulDavisThe1st|2 years ago|reply
When I learned emacs lisp back in the mid-80s, I didn't find any requirement to bend my mind or pay any upfront cost. It was a little different from C (and BASIC), certainly, but once I grasped a few basic concepts, it felt extremely elegant and very well suited to the task(s) to which I was putting it.

More recently, I've felt that way about Python, and about a few chunks of my favorite blobs of self-written C++.

[+] agumonkey|2 years ago|reply
the upfront mental cost debate is still going, a few people tried teachings lisps as first languages and people didn't struggle

i personally cried a few tears when learning java, whereas the weird drscheme class made me all calm and happy

programming also blends a few layers into one, and some people are operating at one (line by line modification of data), some want generic infinite freedom[0], you can rapidly see who will enjoy what there

[0] one trauma i got during studies is failing an exam because after reviewing ada's manual twice, I couldn't find the page explaining the `object'field` syntax and failed not being able to split a string. i never liked ad-hoc syntax much...

[+] linguae|2 years ago|reply
Rust requires people to learn how to adapt to Rust’s borrow checker, which can be challenging for programmers coming from other languages, yet this hasn’t stopped Rust’s rapid adoption in systems programming. In fact, Rust’s safety features are Rust’s selling point.

Of course, it helps that Rust came from Mozilla and that it’s used in Firefox. Other than Grammarly and some internal Google products, I don’t know any modern-day high-profile projects written in Common Lisp. What would help bolster Common Lisp would be a high-profile project that would be much more difficult to implement in other languages.

[+] munchler|2 years ago|reply
I agree with you about the upfront costs. As a functional programming fan (F# in particular), I can’t deny that it sometimes feels like puzzle solving. Fun, but not easy at first.

On the other hand, you can quickly throw together some crappy Python code to do the same thing, but you end up paying 10x the cost over the long run in debugging and rewriting.

So my advice for people who care about the code they write is to pay the upfront cost to do it well. The good news is that the learning curve does get less steep eventually, and at that point you feel like a goddamn ninja.

[+] taeric|2 years ago|reply
This is only true in terms of "free" options. There were (and are?) decent paid options that had quite good tooling. In the free software realm, though, it was lacking.

And... this is kind of the point of the post. The python community doesn't just push to advance only the language, they advance the use of the language in the free software world. What is the closest that the common lisp world has?

[+] BaculumMeumEst|2 years ago|reply
Package management kind of sucks, the ecosystem sucks, other languages continually improve but nobody will ever advance the CL standard. There is nothing compelling about the language to people who aren't already Lisp people.
[+] siknad|2 years ago|reply
> There is nothing compelling about the language to people who aren't already Lisp people.

CL is expression based (like Rust, unlike other mainstream languages I've seen), has a concise macro system (more convenient than Rust's imo), has a GC (simpler to use than languages with manual memory management or RC-only), has a better developed ecosystem then some new languages (ex. automatic ffi generation; while buggy, tremendously helpful compared to writing bindings manually). And it's not pure as in Haskell. And it has type annotations that may be checked at runtime or improve performance. An implementation like SICL could make it viable to use it as a scripting language.

Any similar modern languages with better tooling/ecosystem? Perhaps Julia, haven't seen it yet.

[+] vindarel|2 years ago|reply
For package management, we have:

- Quicklisp, which is pretty handy, since we can install and use a library from the Lisp REPL, without restarting anything.

- Qlot, that installs dependencies locally, and allows to pin them (yeah you can't do that with QL, although cloning a lib and using it is easy).

- ocicl, a new package manager from the world of containers

- CLPM, another new one

- Roswell, if you really really want to install libraries from the terminal (although Qlot does it) or to install implementations, or software.

I bet the ecosystem is bigger than many think: https://github.com/CodyReichert/awesome-cl and anyone would be amazed by its stability.

[+] floren|2 years ago|reply
Package documentation which seems to mostly consist of a list of variables and function names, possibly with a single sentence describing what the function does. Possibly.
[+] agumonkey|2 years ago|reply
it's paradoxical since lisp people weren't born this way, surely there was some appeal (beyond being influenced by teachings) ;)
[+] bsdpufferfish|2 years ago|reply
> nobody will ever advance the CL standard

And that’s a good thing.

[+] kentrado|2 years ago|reply
I love programming in common lisp. It has made programming fun again.

I don't know if that makes me a lisp person. I just know it makes programming fun for me.

[+] kazinator|2 years ago|reply
Popularity is a matter of luck. But suppose that the forces that drive luck somehow aligned themselves with promoting Lisp. There are ways Lisp would sabotage the luck being radiated upon it.

Programmers who get into CL will hit various silly obstacles:

- No standard way to express special characters in string literals.

- No standard Unicode support; no \u1234 notation in the standard. I/O with character encodings is implementation-specific.

- Weird pathname handling that is simultaneously too abstract, and too nonportable, which is oxymoronic. The pathname abstraction caters to features of operating systems that basically no longer exist. Yet at the same time, two CL implementations on the same modern OS (POSIX or Windows) cannot agree on all the details regarding how a pathname string (the real artifact seen by the OS) parses to a pathname object!

- The experience of just firing up a free CL implementation (no Emacs) out of the box and experimenting in its REPL is poor. CLISP has built in history recall and editing; I would recommend that.

- Ironically poor REPL experience in the free implementations, given that Lisp gave us that word. Only CLISP has built in editing and history recall out of the box. Telling newbies they have to learn a specific editor and its Lisp integration is poor and adds to the activation energy.

- Working with objects can be verbose with syntax like (slot-value point 'x) which is just point.x in many other languages. This can be shortened by defining accessors, but accessors abstract a lot. Given (x point) you no longer know that it's just a simple slot. It's a function call, which could be anything. Another thing is, would you give an accessor a one letter name like x, even if it's in a package?

- Newcomers to Lisp do have to learn the list processing. People coming from languages in which lists are collection bags get confused why their list x is not changing after (append x '(1 2)). If you don't use the loop macro and want to collect items into a list, you have to learn the ritual of push-ing items into a stack, and then using nreverse at the end. Guy Steele described a nice procedural list construction API in Common Lisp, The Language, second edition, but it's not in the standard.

- Working with multiple values is verbose, with forms like (multiple-value-bind (quotient reminder) (truncate 1 2) ...), and multiple-value-setq, and whatnot. Some pattern matching or binding libraries help with that.

[+] assimpleaspossi|2 years ago|reply
I started learning Lisp recently. The text I was reading said people complain about the parentheses and I see a number of people stating that here. But that is not a problem when you have an editor that understands lisp and keeps track of all that for you. Just focus on the code and the editor will take care of the parentheses for you.
[+] 2024throwaway|2 years ago|reply
I tried to learn lisp, and this may sound like a trite complaint, but I honestly just could not deal with all the parenthesis. Terrible ergonomics.
[+] Animats|2 years ago|reply
I've written in Common LISP. Here's my port of the Boyer-Moore theorem prover to CLisp.[1] I've used original INTERLISP. I've used a Symbolics LISP machine. I once did a lot of work in Franz LISP.[2] And I'm partially responsible for AutoCAD going with AutoLISP as a scripting language. I've even taken a class from John McCarthy himself, at Stanford.

All that was decades ago. I haven't written a new program in LISP in decades. (In later years Python, Go, Rust, C++, and when forced, JavaScript.)

* Early thinking was that programs needed to be able to modify themselves. This turned out to be a niche feature. It was a holdover from the early days of programming, where storing into the code was considered a normal activity. Von Neumann was into that. To index into an array, load instructions were modified in place. Once index registers were invented, that mostly became unnecessary. Languages can be too dynamic. You pay a price for that. Python suffers from the design feature that anything can store into anything at any time. This blocks most compile-time optimizations.

* Lists as a primitive are OK, but modifying lists all the time isn't what you really want. Most languages have settled on a useful set of collections today, and their implementations, especially hashes, are highly optimized. Python pretty much got this right, and other languages now use roughly Python's various collection types.

* Saving the entire state of the system, instead of having source files, was a really bad idea for anything that had more than one person working on it.

* The overly clever macro system makes code hard to maintain. (I tried to port [2] to Common LISP. Got about half way. It has the original Oppen-Nelson simplifier inside, the first of what's now called a SAT solver. Originally written in MACLISP (MIT's Project MAC, not Apple Macintosh), and ported to Franz LISP by Oppen and Nelson, it has too many clever macros that I wasn't able to completely figure out a decade ago.) This is a generic problem with macro systems, of course, which is why C deliberately had a weak macro system.

LISP is a blast from the path. It's fun for retro reasons, but things have moved on.

[1] https://github.com/John-Nagle/nqthm

[2] https://github.com/John-Nagle/pasv/tree/master/src/CPC4

[+] vindarel|2 years ago|reply
Everyone, if you don't have a clue on how's Common Lisp going these days, I suggest:

https://lisp-journey.gitlab.io/blog/these-years-in-common-li... (https://www.reddit.com/r/lisp/comments/107oejk/these_years_i...)

A curated list of libraries: https://github.com/CodyReichert/awesome-cl

Some companies, the ones we hear about: https://github.com/azzamsa/awesome-lisp-companies/

and oh, some more editors besides Emacs or Vim: https://lispcookbook.github.io/cl-cookbook/editor-support.ht... (Atom/Pulsar support is good, VSCode support less so, Jetbrains one getting good, Lem is a modern Emacsy built in CL, Jupyter notebooks, cl-repl for a terminal REPL, etc)

[+] pfdietz|2 years ago|reply
Back in the day, it required machines that were more powerful than what C programs could run on. By the time machines caught up and Common Lisp was standardized it was too late.
[+] lcall|2 years ago|reply
Somebody famous said something like "Lisp makes hard things easy, and easy things hard", which finally, after all these years, clarified it for me.

(It was quoted on HN in a previous discussion, and I was impressed with the credentials of the person quoted, but I don't have the reference handy.)

[+] harry8|2 years ago|reply
You want to learn programming. Lets get started - how long does that take to bang out 10 exercises? Order the estimated results from quickest to slowest. Here's my estimates (don't like them, show yours! It'll be interesting):

  - Scratch
  - python/perl/ruby/javascript
  - C/Java
  - Swift/Rust
  - Anything else running on the jvm
  - 
  - a whole lot of daylight
  -
  - common lisp.
And many if not most will fail to get to the first exercise, those that do will start it well after all the rest have completed 10.

Land of lisp seemed to be the best learn lisp book and it couldn't even manage to limit itself to one implementation of common lisp that you could use for the entire book.

The situation is ridiculous.

[+] commandlinefan|2 years ago|reply
OTOH, I remember reading somewhere that Tanenbaum was happy when Linux came out so that people would stop asking him to make Minix into a real "usable" operating system. There may be something to be said for letting Lisp remain more conceptual/ideal than practical and allowing other functional languages take on the "warts" associated with being used by everybody.
[+] hcarvalhoalves|2 years ago|reply
A plausible reason: because no big company is backing it with money.

This industry will mostly use whatever Google/Microsoft/Meta sponsors.

[+] iamevn|2 years ago|reply
Personally I'm not especially interested in programming in a language with a separate function namespace. It makes much more sense to me to treat functions more like any other value in the language.
[+] 13415|2 years ago|reply
Generally speaking, the best things are not the most popular. There are probably a few exceptions such as classical music, but I've found this rule of thumb to hold in many different domains.
[+] Nevermark|2 years ago|reply
People’s ability to process complexity must play into where the peak is.

I large culture consisting exclusively of genius composers would no doubt develop styles of music that would relegate our pinnacles of classical music to their “pop”.

No doubt there is a bell curve of how easily each of us intuit novel abstraction, vs. benefit from more predictability (i.e. restrictions, enforced patterns).

Mainstream languages are going to be more predictable and restrictive or patterned, than programming languages auteurs might prefer.