Fantastic news! This is a really interesting place in the design space and has come so close to being lost to history.
I believe the idea is essentially to write C semantics in scheme notation. Variables get marked with 'u32' or similar instead of being implicit sum types of anything the language can represent, memory allocation is explicit instead of garbage collected. In itself that essentially means writing C syntax trees in prefix notation, which is probably an acquired taste.
However scheme also comes with the compile time macro layer and that lot runs just fine in pre-scheme, garbage collected and all, because it's burned off before runtime anyway. Specifically, it's wholly macro-expanded before compilation to C (or similar), which is the obvious lowering to use for execution.
Also scheme has tooling, so if you're careful, the type annotated Cish syntax trees execute correctly as scheme, so you can debug the thing there, unit test it from scheme and so forth.
I really like it as a path to writing lisp runtimes in something that isn't C since an alarming fraction of them turn out to have a C runtime library at the bottom of the stack. Also for writing other things that I tend to write in C, where it's really the semantics I want and the syntax getting in the way.
Smalltalk can use a similar bootstrapping method, which blew my mind the first time I read and understood (see https://ftp.squeak.org/docs/OOPSLA.Squeak.html, "Smalltalk to C Translation"). In that world, images can easily obscure some of the lineage. Scheme has the advantage of having a standard to hold dialects to. I am watching this project with interest.
1. one that runs under a stock Python interpreter (which doesn't use static types)
2. one that's pure C++, translated from statically typed Python code, and from data structures generated in Python
In the second case, everything before main() is "burned off" at build time -- e.g. there is metaprogramming on lexers, flag parsers, dicts, etc. that gets run and then turned into static C data -- i.e. data that incurs zero startup cost
And related to your other point, I remember looking at Racket's implementation around the time it started the Chez Scheme conversion. For some reason, I was surprised that it was over 100K lines of hand-written C in the runtime -- it looked similar to CPython in many ways (which is at least 250K lines of C in the core).
Pre-Scheme is an incredible piece of history, largely forgotten and lost to time outside of a very small group that knew about it. Live hackable at the REPL, and yet with static type inference (Hindley-Milner!), compiles to C, no GC? It's something I've always wanted, and it existed, but it felt like one of those lost pieces of technology that was at risk of fading into the dustbin of history.
But no more! It's so exciting that Andrew Whatson has begun reviving the project with such great enthusiasm and making it so that Pre-Scheme can run on top of a variety of Schemes. And it's wonderful that NLnet has recognized how important this effort is. I think Pre-Scheme could play an interesting role alongside Zed and Rust, and indeed I know that Andrew plans to incorporate many of the newer ideas explored in those languages on top of Pre-Scheme eventually.
Go Pre-Scheme revival... I'm cheering it on, and can't wait to use this stuff myself!
https://github.com/carp-lang/Carp is a recent attempt to create a similar language, but with a Rust-inspired borrow checker. though it looks like prescheme would end up being less dependent on C ultimately, this is another option in the space.
Without a new maintainer and perhaps a group of dedicated supporters, Carp may as well be considered abandoned. The existing head maintainer and major contributor has moved on to other things.
Stumbled upon this a while ago while looking for "a systems Scheme", but abandonware in my book given the most recent commit date (and GH Issue).
Settled on Gerbil Scheme instead, lively community & been actively developed to this day for over 15 years now. Although fair warning, still GC'd and (for now) only type-annotated, not (100%) statically typed. But stdlib-wise and compilation-wise still way more "systems-bent" than most Schemes out there.
I am so excited for a Lispy systems language. Existing languages just don't do it for me, though I think Zig is the closest to being what I'm into. So much good stuff in Scheme48. Glad the good ideas are being revived.
Can recommend Gerbil Scheme. Although fair warning, still GC'd and (for now) only type-annotated, not (100%) statically typed. But stdlib-wise and compilation-wise still way more "systems-bent" than most Schemes out there.
I welcome any and all experiments in the low-level programming space, and writing a good one with Lisp syntax is an obvious approach. Carp (mentioned in the article) was a good start, but seems to have stalled out, and while it's easy to see the advantages of bootstrapping from an ML language, I view it as essential to the philosophy that the compiler itself be in a broadly-compatible Lisp syntax, compatible, that is, to the resulting language or sub-language.
But I also think it will exacerbate an existing problem with C, namely, macros. Low-level programming is all about knowing exactly what's going on, and since C has a preprocessor, that's more difficult than it otherwise would be. Just because something looks like a function call, doesn't mean it actually is one.
Schemes have a much better macro system, and that will simultaneously make the core issue both better, and worse. But it's very much worthwhile to try it, imho, and see if good tooling can ameliorate the downsides, while still enjoying the power, and freedom from tedium, which macros bring to the table.
Scheme with HM type system sounds fun. I've used ocaml a fair bit and I really find that the sweet spot for effectiveness of types vs arguing with the compiler. Racket and common lisp both have optional type systems but neither ever really clicked with me.
few days I've spent a few hours with prescheme, but was stopped in the end because there where gcc errors. I've felt a little guilty to have have spent some quite some time, but not have achieved anything. But thats the thing with Prescheme, so fascinating I could not resist. I mean look at its history, at all the cool and unique features.
Anyway, very cool to read this news, I'm also a little relieved not to have burnt all this hours into nothings. Very glad to see the story continues.
> On another front, the Guix project is a major force bringing new users to Scheme, providing an unparalleled foundation for free and reproducible computing.
The Nix/OS folks might take exception. I'm guessing this is tongue-in-cheek but it belies the tone of the rest of the post.
In all seriousness, though, this is exciting from a modern, end-user's vantage point and fascinating from an historical perspective.
Nixpkgs also doesn't seem to require that all packages be built from source - which, if you're really looking for reproducibility, is a downside. I recognize that there are practical reasons for this, and it's part of why Nix has so many more packages available than Guix, but IMO it makes Guix a better foundation to build on if you want as much of your system as possible to be reproducible.
This is very cool. I've added it to my RSS reader, can't wait to see what comes of it.
Genuine question: would there be any advantages in targeting LLVM IR, rather than transpiling to C? With C being notoriously implementation dependent (down to things like the sizes of integer types), it seems like a messy target for something intended to be a sane systems language.
Targeting LLVM IR has the drawback that it is not platform independent: Details of calling conventions must be modeled in the IR, so the compiler must know what ABI it is targeting and emit the appropriate code. Compiling to C doesn't have this problem, since the C compiler will handle calling conventions for you.
That said, LLVM would indeed have some advantages. Scheme has guaranteed tail call optimization, which you cannot guarantee with C. But LLVM does allow you to annotate calls as tail calls, and it can transform tail self-recursion into a loop for you.
According to the article: thanks to a grant from the NLnet foundation under the NGI Zero Core program, Pre-Scheme can continue to be developed. It's supposed to be a C alternative. Currently, it compiles to C, has a Hindley Milner type system, macros, and it can run in a Scheme REPL. And they have a roadmap of features now.
This is pretty cool, and it's generous of them to grant them funding, but (and I'm not trying to be rude) I wonder why they chose to give a grant for Pre-Scheme specifically. This seems only loosely related to the goals of the NGI Zero Core program (linked in the article):
"The next generation internet initiative envisions the information age will be an era that brings out the best in all of us. We want to enable human potential, mobility and creativity at the largest possible scale – while dealing responsibly with our natural resources. In order to preserve and expand the European way of life, the programme helps shape a value-centric, human and inclusive Internet for all."
...
"We want a more resilient, trustworthy and open internet. We want to empower end-users. Given the speed at which the 'twin transition' is taking place, we need a greener internet and more sustainable services sooner rather than later. Neither will happen at global scale without protocol evolution, which — as the case of three decades of IPv6 introduction demonstrates — is extremely challenging. NGI0 Core is designed to push beyond the status quo and create a virtuous cycle of innovation through free and open source software, libre hardware and open standards. If we want everyone to use and benefit from the internet to its full potential without holding back, the internet must be built on strong and transparent technologies that allow for permissionless innovation and are equally accessible to all."
Pre-Scheme is one more path to moving away from low level programming being done on top of directly programmed C for one thing, and the revival effort directly ties in by moving Pre-Scheme to be on top of r7rs, an open standard. This opens up Pre-Scheme to a variety of other ecosystems that NLnet already invests in, including Guix, Mes, and Guile, which have put a lot of efforts into secure and highly reproducible (and indeed bootstrappable) computing. There's definitely some ties in with security and security-oriented communities NLnet already funds, and this project directly works towards moving towards a more standardized approach, leading to hopefully broader adoption.
Another way to look at it is a long shot on a new higher level language to entice the folks currently reaching for Java, Python, or other runtime interpreters. An easy and safe language that shifts more work to compile time and offers high performance with ergonomic use will use less power, reduce the education requirements to writing fast code, and potentially change the world like another fast memory safe language governments are increasingly endorsing. Or if nothing else it may serve as a language for interpreters of those languages, or to make extensions to them more ergonomic. With enough scale the improvements can come very incrementally.
NLNet is doing god's work funding really cool projects that would have a very hard time justifying their existence to some mainstream donors.
I dream of some day soon running Emacs/Guix/Hurd on an open RISC-V chip and not having it be some flossy novelty but a genuine spiritual successor to Genera and the Lisp Machines.
Tail-call optimization is very important when writing Scheme programs. By removing those, you loose the power of recursion.
Also when it comes to macros, does that include `syntax-rules` or `syntax-case` style macros, where the latter are much more powerful?
While an embedded Scheme-like language is incredibly useful, at some point I feel as if you would simply have to include these features, and to that end it would just be Scheme reinvented.
Please don't write "thanks to a generous grant from the NLnet foundation under the NGI Zero Core program". The most of money comes from the European Commission through the Horizon Europe / NGI funding schemes. NLNet is mainly the operator of the call.
The European Commission deserves thanks for funding the commons, a thing that rarely happens by governments, but should! NGI Zero was thanked though?
NLnet being the operator of the call is no small thing though, having been through the process they are very thoughtful, knowledgeable, thorough in how they run things. They even run the software they fund and verify it's working and check that the overall ideas are sensible, which is something I can't say of many other grant programs I've interacted with. So NLnet does deserve thanks.
you can read that not as that Scheme prefix syntax alone is a big selling point, but the fact that it then supports Scheme macros (which are much better than in most other languages that support some kind of macros, partly due to the syntax making this easier).
Then you can read the rest of the sentence, for a bonus:
> and a compatibility library to run Pre-Scheme code in a Scheme interpreter.
Which means that you can do things like develop using this language within a normal Scheme development environment, possibly share code between developing for the PreScheme compiler target and non-PreScheme targets, etc.
Pre-Scheme can be run in a Scheme system, then when it is found to be correct there, compiled VERY straightforwardly to C. This is a huge win in terms of productivity. Plus, at the top level, at compile time, you have all of Scheme available.
JonChesterfield|1 year ago
I believe the idea is essentially to write C semantics in scheme notation. Variables get marked with 'u32' or similar instead of being implicit sum types of anything the language can represent, memory allocation is explicit instead of garbage collected. In itself that essentially means writing C syntax trees in prefix notation, which is probably an acquired taste.
However scheme also comes with the compile time macro layer and that lot runs just fine in pre-scheme, garbage collected and all, because it's burned off before runtime anyway. Specifically, it's wholly macro-expanded before compilation to C (or similar), which is the obvious lowering to use for execution.
Also scheme has tooling, so if you're careful, the type annotated Cish syntax trees execute correctly as scheme, so you can debug the thing there, unit test it from scheme and so forth.
I really like it as a path to writing lisp runtimes in something that isn't C since an alarming fraction of them turn out to have a C runtime library at the bottom of the stack. Also for writing other things that I tend to write in C, where it's really the semantics I want and the syntax getting in the way.
jaccarmac|1 year ago
fire_lake|1 year ago
chubot|1 year ago
There are two complete implementations
1. one that runs under a stock Python interpreter (which doesn't use static types)
2. one that's pure C++, translated from statically typed Python code, and from data structures generated in Python
In the second case, everything before main() is "burned off" at build time -- e.g. there is metaprogramming on lexers, flag parsers, dicts, etc. that gets run and then turned into static C data -- i.e. data that incurs zero startup cost
Comparison to Pre-Scheme: https://lobste.rs/s/tjiwrd/revival_pre_scheme_systems_progra... (types, compiler output, and GC)
Brief Descriptions of a Python to C++ Translator - https://www.oilshell.org/blog/2022/05/mycpp.html
...
And related to your other point, I remember looking at Racket's implementation around the time it started the Chez Scheme conversion. For some reason, I was surprised that it was over 100K lines of hand-written C in the runtime -- it looked similar to CPython in many ways (which is at least 250K lines of C in the core).
paroneayea|1 year ago
But no more! It's so exciting that Andrew Whatson has begun reviving the project with such great enthusiasm and making it so that Pre-Scheme can run on top of a variety of Schemes. And it's wonderful that NLnet has recognized how important this effort is. I think Pre-Scheme could play an interesting role alongside Zed and Rust, and indeed I know that Andrew plans to incorporate many of the newer ideas explored in those languages on top of Pre-Scheme eventually.
Go Pre-Scheme revival... I'm cheering it on, and can't wait to use this stuff myself!
voidhorse|1 year ago
dleslie|1 year ago
https://github.com/carp-lang/Carp/issues/1460#issuecomment-2...
dualogy|1 year ago
Settled on Gerbil Scheme instead, lively community & been actively developed to this day for over 15 years now. Although fair warning, still GC'd and (for now) only type-annotated, not (100%) statically typed. But stdlib-wise and compilation-wise still way more "systems-bent" than most Schemes out there.
davexunit|1 year ago
dualogy|1 year ago
Can recommend Gerbil Scheme. Although fair warning, still GC'd and (for now) only type-annotated, not (100%) statically typed. But stdlib-wise and compilation-wise still way more "systems-bent" than most Schemes out there.
samatman|1 year ago
But I also think it will exacerbate an existing problem with C, namely, macros. Low-level programming is all about knowing exactly what's going on, and since C has a preprocessor, that's more difficult than it otherwise would be. Just because something looks like a function call, doesn't mean it actually is one.
Schemes have a much better macro system, and that will simultaneously make the core issue both better, and worse. But it's very much worthwhile to try it, imho, and see if good tooling can ameliorate the downsides, while still enjoying the power, and freedom from tedium, which macros bring to the table.
giraffe_lady|1 year ago
Y_Y|1 year ago
https://lexi-lambda.github.io/hackett/
brabel|1 year ago
[1] https://coalton-lang.github.io/
dang|1 year ago
Pre-scheme: A Scheme dialect for systems programming (1997) [pdf] - https://news.ycombinator.com/item?id=29725313 - Dec 2021 (12 comments)
(surprised there hasn't been more)
roleks|1 year ago
ethagnawl|1 year ago
The Nix/OS folks might take exception. I'm guessing this is tongue-in-cheek but it belies the tone of the rest of the post.
In all seriousness, though, this is exciting from a modern, end-user's vantage point and fascinating from an historical perspective.
ryukafalz|1 year ago
Nixpkgs also doesn't seem to require that all packages be built from source - which, if you're really looking for reproducibility, is a downside. I recognize that there are practical reasons for this, and it's part of why Nix has so many more packages available than Guix, but IMO it makes Guix a better foundation to build on if you want as much of your system as possible to be reproducible.
troad|1 year ago
Genuine question: would there be any advantages in targeting LLVM IR, rather than transpiling to C? With C being notoriously implementation dependent (down to things like the sizes of integer types), it seems like a messy target for something intended to be a sane systems language.
gergo_barany|1 year ago
Targeting LLVM IR has the drawback that it is not platform independent: Details of calling conventions must be modeled in the IR, so the compiler must know what ABI it is targeting and emit the appropriate code. Compiling to C doesn't have this problem, since the C compiler will handle calling conventions for you.
That said, LLVM would indeed have some advantages. Scheme has guaranteed tail call optimization, which you cannot guarantee with C. But LLVM does allow you to annotate calls as tail calls, and it can transform tail self-recursion into a loop for you.
trealira|1 year ago
This is pretty cool, and it's generous of them to grant them funding, but (and I'm not trying to be rude) I wonder why they chose to give a grant for Pre-Scheme specifically. This seems only loosely related to the goals of the NGI Zero Core program (linked in the article):
"The next generation internet initiative envisions the information age will be an era that brings out the best in all of us. We want to enable human potential, mobility and creativity at the largest possible scale – while dealing responsibly with our natural resources. In order to preserve and expand the European way of life, the programme helps shape a value-centric, human and inclusive Internet for all."
...
"We want a more resilient, trustworthy and open internet. We want to empower end-users. Given the speed at which the 'twin transition' is taking place, we need a greener internet and more sustainable services sooner rather than later. Neither will happen at global scale without protocol evolution, which — as the case of three decades of IPv6 introduction demonstrates — is extremely challenging. NGI0 Core is designed to push beyond the status quo and create a virtuous cycle of innovation through free and open source software, libre hardware and open standards. If we want everyone to use and benefit from the internet to its full potential without holding back, the internet must be built on strong and transparent technologies that allow for permissionless innovation and are equally accessible to all."
paroneayea|1 year ago
392|1 year ago
Y_Y|1 year ago
I dream of some day soon running Emacs/Guix/Hurd on an open RISC-V chip and not having it be some flossy novelty but a genuine spiritual successor to Genera and the Lisp Machines.
a2code|1 year ago
Also when it comes to macros, does that include `syntax-rules` or `syntax-case` style macros, where the latter are much more powerful?
While an embedded Scheme-like language is incredibly useful, at some point I feel as if you would simply have to include these features, and to that end it would just be Scheme reinvented.
dg_meme|1 year ago
paroneayea|1 year ago
NLnet being the operator of the call is no small thing though, having been through the process they are very thoughtful, knowledgeable, thorough in how they run things. They even run the software they fund and verify it's working and check that the overall ideas are sensible, which is something I can't say of many other grant programs I've interacted with. So NLnet does deserve thanks.
tmtvl|1 year ago
hayley-patton|1 year ago
neilv|1 year ago
> Scheme syntax, with full support for macros,
you can read that not as that Scheme prefix syntax alone is a big selling point, but the fact that it then supports Scheme macros (which are much better than in most other languages that support some kind of macros, partly due to the syntax making this easier).
Then you can read the rest of the sentence, for a bonus:
> and a compatibility library to run Pre-Scheme code in a Scheme interpreter.
Which means that you can do things like develop using this language within a normal Scheme development environment, possibly share code between developing for the PreScheme compiler target and non-PreScheme targets, etc.
bitwize|1 year ago
kagevf|1 year ago
Zambyte|1 year ago
pjc50|1 year ago