Having played around with Clojure and Scheme for a while (but never got too serious), I always thought homoiconicity and macros were extremely cool concepts, but I never actually ran into a need for them in my everyday work.
>Now, if we agree that the ability to manipulate source code is important to us, what kind of languages are most conducive for supporting it?
This is useful for compiler programmers, or maybe also those writing source code analyzers/optimizers, but is that it? On occasion I have had to write DSLs for the user input, but in these cases the (non-programmer) users didn't want to write Lisp so I used something like Haskell's parsec to parse the data.
The remote code example given in the post is compelling, but again seems a bit niche. I don't doubt that it's sometimes useful but is it reason enough to choose the language? Are there examples of real-life (non-compilers-related) Lisp programs that show the power of homoiconicity?
Same goes with the concept of "being a guest" in the programming language. I have never wanted to change "constant" to "c". Probably I'm not imaginative enough, but this has never really been an issue for me. Perhaps it secretly has though, and some of my problems have been "being a guest" in disguise.
In my experience, macros in lisp codebases are mainly abstracting away common boilerplate patterns. Common Lisp has WITH-OPEN-FILE to open a file and make sure it closes if the stack unwinds. This is a macro based on the UNWIND-PROTECT primitive, which ensures execution of a form if the stack unwinds. Many projects will have various with-foo macros to express this pattern for arbitray things that are useful in the context of that project(though not all with- macros need UNWIND-PROTECT). Another example is custom loops that happen over and over.
Let's say I'm writing a chess engine. I regularly want to iterate over the legal moves in a position. So I might make a macro so I can write
(do-moves (mv pos) ...)
I find that because doing this is so simple, well written lisp codebases tend to be pretty easy to read. There's less of that learning curve in a new codebase of getting used to the codebase's particular boilerplate rituals, and learning to pick out the code that's interesting. In a good lisp codebase all the ritual is hidden away in self-documenting macros.
Of course this can get taken too far and then it becomes nightmarish to understand the codebase because so much stuff is in various complex macros that it's hard to tell where the shit goes down.
I've been using Java since version 1.1 and I've seen features added that required a new version of the language spec to go through committee and get implemented before we got to use them where you could just add these features yourself at the top of any Lisp file and then immediately start using them. For example consider the try-with-resources [0] syntax sugar. So instead of thinking "I never actually ran into a need for them", think of new language features that you have started using: those are the kinds of things you could've added yourself.
Also look at any kind of code-gen tooling like parser generators or record specifications like Protocol Buffers as examples of what you could do within the language.
> This is useful for compiler programmers, or maybe also those writing source code analyzers/optimizers, but is that it? On occasion I have had to write DSLs for the user input, but in these cases the (non-programmer) users didn't want to write Lisp so I used something like Haskell's parsec to parse the data.
If you're talking about Haskell, you should be talking about folk who write template Haskell, which is the macro system for GHC. There are plenty of Haskell programmers who know how to write Template Haskell, and there are plenty of Haskell programmers who don't. By contrast, I don't think there's a single Lisp programmer who can't write Lisp macros.
That's homoiconicity. Once you learn Lisp, you automatically know how to write Lisp macros. Once you learn Haskell (or Ocaml or Rust), you don't automatically know how to write macros in that language (and the macro system may not even be portable across compilers).
Now, stuff that would have to be implemented in the compiler to update the language, can now be written just as a "normal" program, that adds whole new features to your programming language.
For example, the entire object system in Common Lisp was implemented as macros.
Yes, most programming tasks don't require this kind of power. But it does mean programming in Lisp it's very very rare you are going to be stuck because your programming language doesn't implement some feature you need for the task at hand.
I actively want people to NOT change constant to c. I want the language to be a predictable shared base, not something I have to relearn and customize for every project.
I'm not a language designer. That's hard. That takes time and effort. And I don't do anything that can't be done in Python or Dart or whatever. Customizing the language is time not spent on the project, and batteries included stuff already has everything I need.
I think lisp is good for people who "think inside their heads", as in, they think "I want to do this, oh, I could do it this way, then I'd need this resource, let's build it".
If you think "Interactively" as in "I want to do this, what does Google tell me others are doing, oh yeah, this was made almost exactly the same use case, I'll start with this resource, now I'll adapt my design to fit it", you might not have any ideas for language tweaks to make in the first place.
I basically never code and think "I wish I could do that in this language" aside from minor syntactic sugar and library features. New abstractions and ideas don't just pop up in my mind, what the common mainstream languages have is the entirity of what programs are made of, as far as I'm concerned.
> This is useful for compiler programmers, or maybe also those writing source code analyzers/optimizers, but is that it?
It is also useful for anyone wanting to implement language-level features as simple libraries. Someone else brought up Nim here: it's a great example of what can be done with metaprogramming (and in a non-Lisp language) as it intentionally sticks to a small-but-extendable-core design.
There's macro-based libraries that implement the following, with all the elegance of a compiler feature: traits, interfaces, classes, typeclasses, contracts, Result types, HTML (and other) DSLs, syntax sugar for a variety of things (notably anonymous functions `=>` and Option types `?`), pattern matching (now in the compiler), method cascading, async/await, and more that I'm forgetting.
As part of delivering an e-commerce solution in Scheme, I wrote a module which allowed for SQL queries to be written in s-expression syntax. You could unquote Scheme variables or expressions into the queries and it would do the right thing wrt quoting, so no connection.prepareStatement("INSERT INTO CUSTOMERS (NAME, ADDRESS) VALUES (?, ?)", a, b) type madness. Wanky, you bet. But oh so, so convenient.
It's not tied to complex DSLs and compilation. I was frustrated by the lack of f-strings in emacs lisp so I hacked a macro, now I have embedded scoped string interpolation. I can write (f "<{class} {fields}>"). Having these freedom is really not a niche, it's mentally and pragmatically important.
Most well-written Lisp code has its structure indicated primarily by the indentation. Some form of offset rule is the most commonly suggested alternative Lisp syntax. It does make the tree structure more obvious:
The left looks a lot like a typical AST for many languages, such as C, shortly after parsing. As the article points out, it's the easy access to this tree-list structure at runtime, that can be easily handled as data by the program itself, that gets Lisp programmers so excited.
> The left looks a lot like a typical AST for many languages....
Right, and this is what's meant by the oft-repeated statement "LISP has no syntax." Of course it has syntax in the sense that if you were to go in and delete one of those parens, the whole thing wouldn't compile, but the syntax it does have is 100% regular.
You could stack them like this in every language but only in Lisp do people actually do it. Lisps bad habbit of stacking all the parentheses like that is what makes it so hard to read. It is easier to write that way, but it is very hard to understand how many contexts up you just moved.
Same here. I never understood why a paren on the left of a function name “looks like fingernail clippings” but to the right it’s “just how code works.”
I was personally blown away when I read PG's summary of McCarthy's paper where he came up with a few primitives and built literally any function based on them, including the `eval' function itself. All of this based on some really basic primitives like `atom' and `lambda'.
If writing `eval' is possible in the same language you started with, then you must be able to write your own eval, no? Or redefine the primitives? But then what happens if you accidentally set DEFINE to be 5? Restart the buffer? Is there any way to get back to your original definition of DEFINE after corrupting it?
I was gonna respond that you can't redefine the primitives because they're not functions or macros but special forms. And this seems true in Common Lisp. I can't define a function named SETQ(the primitive for assignment in CL, also called set! in other lisps) because it's a reserved name. But then I thought I'd try it in Guile and there I was allowed to (define define 5). Even though define is also a special form in Guile. Which made me unable to define further things, telling me 5 is not a valid argument to APPLY. And no I don't know how to fix it...
Very great presentation! I made a little toy some time ago [1] with YAML files compiled into Javascript code, in Lisp style, basically along the same lines as the article, and I use this project to manage my dotfiles. The basic syntax probably looks like this:
Lisp is inspiring at first glance, but when you need to solve complex problems like references, pointers, macros, byte-compilation and native compilation it just is not expressive enough like C, C++, or Rust. Neither is Javascript. Lisp could not replace all other languages
I seriously can't downvote your comment hard enough.
> references
Everything in Lisp is passed by copied references (when talking in C++ terms), so that's kind of a solved problem.
> pointers
You don't have raw pointers in CL since it's a memory-managed language, unless you're dealing with foreign code, and then there's support for handling these in a meaningful way (see CFFI).
> macros
Seriously? The CL macro system with quote-unquote and the full language being available has been good enough to be an inspiration for syntactic macro systems in multiple other languages, such as Rust or Elixir.
> byte-compilation
You only compile to bytecode if you cannot compile to native code. You can see it in implementations like ECL (if GCC is not present) and CLISP (if not built to use GNU Lightning).
> native compilation
SBCL and CCL and ECL and Clasp and LispWorks and ACL all do that, it's a solved problem. Or do you mean that writing compilers in Lisp is impossible since the language is not expressive enough, at which point you can see the source code for Lisp compilers which is frequently written in Lisp?
> Neither is Javascript.
That's the only part of your comment that I think makes any sense.
> Lisp could not replace all other languages.
Who even posed the statement that it should? Are you trolling?
This is entirely inaccurate and a fundamental misunderstanding of Lisp, both as a language and as an ecosystem. Have you seen [0], for example? It's how to leverage the native assembler of a Common Lisp compiler to build a small JIT-like compiler in Common Lisp.
Did you know Lisp has a disassembled built-in, with the standard library function DISASSEMBLE [1]?
Why would Lisp support these things like "native compilation" are supposedly out of scope?
Never mind that Lisp solved references, pointers, macros, byte-compilation and native compilation before the calendar flipped from 1969.
Lisp was compiled by around 1960.
Peter Landing, in 1964, presented the SECD machine, a kind of bytecode based on Lisp-like lists, directly usable for Lisp implementations, in "The Mechanical Evaluation of Expressions". Numerous Lisps have historically had some kind of "Lisp Assembly Language" (LAP) as an alternative or in addition to native compilation.
ZetaLisp, in 1981, the variant of MacLisp running on Lisp machines, featured locatives, which allow the address of a location to be passed around, for simulating reference parameters or members.
What do you mean? Many lisps have these features, in fact lisps in general are well-regarded for their macro systems. I don't think any programming language can replace all other languages but of all options, I feel lisps which allow you to safely and cleanly extend the language to better reflect the problem domain cast an unusually wide shadow.
[+] [-] reifyx|3 years ago|reply
>Now, if we agree that the ability to manipulate source code is important to us, what kind of languages are most conducive for supporting it?
This is useful for compiler programmers, or maybe also those writing source code analyzers/optimizers, but is that it? On occasion I have had to write DSLs for the user input, but in these cases the (non-programmer) users didn't want to write Lisp so I used something like Haskell's parsec to parse the data.
The remote code example given in the post is compelling, but again seems a bit niche. I don't doubt that it's sometimes useful but is it reason enough to choose the language? Are there examples of real-life (non-compilers-related) Lisp programs that show the power of homoiconicity?
Same goes with the concept of "being a guest" in the programming language. I have never wanted to change "constant" to "c". Probably I'm not imaginative enough, but this has never really been an issue for me. Perhaps it secretly has though, and some of my problems have been "being a guest" in disguise.
[+] [-] mtlmtlmtlmtl|3 years ago|reply
Let's say I'm writing a chess engine. I regularly want to iterate over the legal moves in a position. So I might make a macro so I can write (do-moves (mv pos) ...)
I find that because doing this is so simple, well written lisp codebases tend to be pretty easy to read. There's less of that learning curve in a new codebase of getting used to the codebase's particular boilerplate rituals, and learning to pick out the code that's interesting. In a good lisp codebase all the ritual is hidden away in self-documenting macros.
Of course this can get taken too far and then it becomes nightmarish to understand the codebase because so much stuff is in various complex macros that it's hard to tell where the shit goes down.
[+] [-] haroldl|3 years ago|reply
Also look at any kind of code-gen tooling like parser generators or record specifications like Protocol Buffers as examples of what you could do within the language.
[0] https://docs.oracle.com/javase/tutorial/essential/exceptions...
[+] [-] momentoftop|3 years ago|reply
If you're talking about Haskell, you should be talking about folk who write template Haskell, which is the macro system for GHC. There are plenty of Haskell programmers who know how to write Template Haskell, and there are plenty of Haskell programmers who don't. By contrast, I don't think there's a single Lisp programmer who can't write Lisp macros.
That's homoiconicity. Once you learn Lisp, you automatically know how to write Lisp macros. Once you learn Haskell (or Ocaml or Rust), you don't automatically know how to write macros in that language (and the macro system may not even be portable across compilers).
[+] [-] jimbokun|3 years ago|reply
But that's incredibly powerful.
Now, stuff that would have to be implemented in the compiler to update the language, can now be written just as a "normal" program, that adds whole new features to your programming language.
For example, the entire object system in Common Lisp was implemented as macros.
Yes, most programming tasks don't require this kind of power. But it does mean programming in Lisp it's very very rare you are going to be stuck because your programming language doesn't implement some feature you need for the task at hand.
[+] [-] eternityforest|3 years ago|reply
I'm not a language designer. That's hard. That takes time and effort. And I don't do anything that can't be done in Python or Dart or whatever. Customizing the language is time not spent on the project, and batteries included stuff already has everything I need.
I think lisp is good for people who "think inside their heads", as in, they think "I want to do this, oh, I could do it this way, then I'd need this resource, let's build it".
If you think "Interactively" as in "I want to do this, what does Google tell me others are doing, oh yeah, this was made almost exactly the same use case, I'll start with this resource, now I'll adapt my design to fit it", you might not have any ideas for language tweaks to make in the first place.
I basically never code and think "I wish I could do that in this language" aside from minor syntactic sugar and library features. New abstractions and ideas don't just pop up in my mind, what the common mainstream languages have is the entirity of what programs are made of, as far as I'm concerned.
[+] [-] j-james|3 years ago|reply
It is also useful for anyone wanting to implement language-level features as simple libraries. Someone else brought up Nim here: it's a great example of what can be done with metaprogramming (and in a non-Lisp language) as it intentionally sticks to a small-but-extendable-core design.
There's macro-based libraries that implement the following, with all the elegance of a compiler feature: traits, interfaces, classes, typeclasses, contracts, Result types, HTML (and other) DSLs, syntax sugar for a variety of things (notably anonymous functions `=>` and Option types `?`), pattern matching (now in the compiler), method cascading, async/await, and more that I'm forgetting.
https://github.com/ringabout/awesome-nim#language-features
[+] [-] TylerE|3 years ago|reply
E.g. sees Nim's take on it: https://nim-lang.org/docs/tut3.html
[+] [-] bitwize|3 years ago|reply
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] agumonkey|3 years ago|reply
[+] [-] zmgsabst|3 years ago|reply
[+] [-] retrac|3 years ago|reply
[+] [-] bryanlarsen|3 years ago|reply
https://readable.sourceforge.io/
[+] [-] amelius|3 years ago|reply
[+] [-] actually_a_dog|3 years ago|reply
Right, and this is what's meant by the oft-repeated statement "LISP has no syntax." Of course it has syntax in the sense that if you were to go in and delete one of those parens, the whole thing wouldn't compile, but the syntax it does have is 100% regular.
[+] [-] Jensson|3 years ago|reply
You could stack them like this in every language but only in Lisp do people actually do it. Lisps bad habbit of stacking all the parentheses like that is what makes it so hard to read. It is easier to write that way, but it is very hard to understand how many contexts up you just moved.
[+] [-] praptak|3 years ago|reply
[+] [-] bkirkbri|3 years ago|reply
[+] [-] eatonphil|3 years ago|reply
An Intuition for Lisp Syntax (stopa.io) 130 points by codetrotter on May 27, 2021 | hide | past | favorite | 114 comments
https://news.ycombinator.com/item?id=27307388
[+] [-] Jtsummers|3 years ago|reply
https://news.ycombinator.com/item?id=24892297 - Oct 26, 2020 (198 comments)
[+] [-] behnamoh|3 years ago|reply
If writing `eval' is possible in the same language you started with, then you must be able to write your own eval, no? Or redefine the primitives? But then what happens if you accidentally set DEFINE to be 5? Restart the buffer? Is there any way to get back to your original definition of DEFINE after corrupting it?
[+] [-] mtlmtlmtlmtl|3 years ago|reply
[+] [-] easeout|3 years ago|reply
[+] [-] theowenyoung|3 years ago|reply
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] orthoxerox|3 years ago|reply
[+] [-] 100001_100011|3 years ago|reply
And there lies the solution: fix eval.
[+] [-] qsort|3 years ago|reply
[+] [-] cjohansson|3 years ago|reply
[+] [-] phoe-krk|3 years ago|reply
> references
Everything in Lisp is passed by copied references (when talking in C++ terms), so that's kind of a solved problem.
> pointers
You don't have raw pointers in CL since it's a memory-managed language, unless you're dealing with foreign code, and then there's support for handling these in a meaningful way (see CFFI).
> macros
Seriously? The CL macro system with quote-unquote and the full language being available has been good enough to be an inspiration for syntactic macro systems in multiple other languages, such as Rust or Elixir.
> byte-compilation
You only compile to bytecode if you cannot compile to native code. You can see it in implementations like ECL (if GCC is not present) and CLISP (if not built to use GNU Lightning).
> native compilation
SBCL and CCL and ECL and Clasp and LispWorks and ACL all do that, it's a solved problem. Or do you mean that writing compilers in Lisp is impossible since the language is not expressive enough, at which point you can see the source code for Lisp compilers which is frequently written in Lisp?
> Neither is Javascript.
That's the only part of your comment that I think makes any sense.
> Lisp could not replace all other languages.
Who even posed the statement that it should? Are you trolling?
[+] [-] dreamcompiler|3 years ago|reply
[+] [-] reikonomusha|3 years ago|reply
Did you know Lisp has a disassembled built-in, with the standard library function DISASSEMBLE [1]?
Why would Lisp support these things like "native compilation" are supposedly out of scope?
[0] https://pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembly-co...
[1] http://www.lispworks.com/documentation/lw70/CLHS/Body/f_disa...
[+] [-] kazinator|3 years ago|reply
Lisp was compiled by around 1960.
Peter Landing, in 1964, presented the SECD machine, a kind of bytecode based on Lisp-like lists, directly usable for Lisp implementations, in "The Mechanical Evaluation of Expressions". Numerous Lisps have historically had some kind of "Lisp Assembly Language" (LAP) as an alternative or in addition to native compilation.
ZetaLisp, in 1981, the variant of MacLisp running on Lisp machines, featured locatives, which allow the address of a location to be passed around, for simulating reference parameters or members.
[+] [-] iamevn|3 years ago|reply
[+] [-] blain_the_train|3 years ago|reply
if you need to build things that build the building blocks of lists it's not a good fit.