top | item 38508779

Steel – An embeddable and extensible Scheme dialect

245 points| MaximilianEmel | 2 years ago |github.com

184 comments

order
[+] mattparas|2 years ago|reply
This is a surprise! Steel is my project, happy to answer any questions anyone might have
[+] dataangel|2 years ago|reply
Any chance of seeing the things I miss the most from Racket in other langs?

1. Parameters and Syntax Parameters (Syntax parameters make macros more powerful)

2. Turing complete macros (not just syntax-case)

3. Typed Racket

I almost used https://gamelisp.rs/ for a project but the nightly feature it needs broke and it's no longer maintained, glad to see something similar arise! You might want to consider adopting their choice of VecDeque as a list replacement, I think it makes a lot more sense than naive linked lists on modern machines.

[+] duttakapil|2 years ago|reply
May I ask, what is your personal journey of learning to code? Did you also discover Lisp/Scheme through SICP? And what have you professionally used Scheme for?

I am currently going through SICP, and I am also interested in Rust, so this project is a great discovery! Maybe I can contribute to it also.

[+] JonChesterfield|2 years ago|reply
What are the immutable hash maps based on? How do you use functions or other maps as keys in them?
[+] SushiHippie|2 years ago|reply
Can someone explain these scheme and lisp languages to me? Every time I look at these languages, I can't grasp what you can use them for. And why one would use them.

I always feel like I'm missing something. In the example scripts and code snippets, I can see that you can define functions, that you can use lists, mathematical operations, you can build some algorithms, you can print text, but it never goes further than that.

I've only used languages like Python, Rust, C, Java, JavaScript, and they all have a very similar vibe, you have a std lib, which can interact with many things, you can build UIs, networking libraries and all that. And I could probably start using any language that is "similar" to these.

But I could never use one of these scheme/lisp languages, as I can't really grasp them.

Sorry, this comment is all over the place, because I can't really explain what's going on in my head when I see languages like this. I'd call myself a proficient programmer, but every time I look at these languages, it feels like as I've never seen code once in my lifetime.

Any help or hint at what I'm missing, is appreciated.

[+] trenchgun|2 years ago|reply
> you have a std lib, which can interact with many things, you can build UIs, networking libraries and all that.

Racket has all of that: https://docs.racket-lang.org

Common Lisp too: https://common-lisp.net/libraries

Clojure too: https://www.clojure-toolbox.com

(Or you could even just build anything within Emacs with Emacs Lisp: https://en.wikipedia.org/wiki/Emacs_Lisp)

> And I could probably start using any language that is "similar" to these.

Then you probably could start also using any of the Lisps mentioned. Most likely you are just hung up on the surface syntax, which does not take long to get used to.

[+] neilv|2 years ago|reply
> it feels like as I've never seen code once in my lifetime.

I think you must be fixating on syntax, or have been seeing some code that is advanced or (it exists) a very confusing example by some academic to demonstrate some curiosity.

Except for idiomatic recursion (which you don't have to use), Scheme semantics should initially look familiar to a Python or JavaScript programmer, like a subset of that, just with a different syntax. (Scheme nuances are much better designed, but to a new programmer semantics will look like a subset.)

And the Scheme syntax is one of the simplest ever, once you understand it.

What should instead be confusing is a language with very different semantics, like lazy evaluation, or an OOPL with complicating dispatch rules to reason about.

[+] brundolf|2 years ago|reply
Most of them use most of the same core idioms you're familiar with from other languages, they just look superficially different because of the syntax (especially the parenthesis placement)

Try moving the left parenthesis of each expression over to the right by one and it may become clearer

(my-fn 1 2)

my-fn (1 2)

[+] globular-toast|2 years ago|reply
Why not just program everything in assembly? High level languages give you powers of expression that lead to better programs or are simply more convenient for the particular task at hand. Lisp languages have several features that aren't found in most other programming languages. And a lot features that are found in other languages originated in Lisp.

Perhaps what you are missing is the practical part. For that you should look to the two types of Lisp in widespread usage: Common Lisp and Emacs Lisp. Common Lisp is a general-purpose language with a very large standard library and rich set of third-party libraries. Emacs Lisp is a complete language, but only really used to build text editing like stuff for Emacs. There's tons of real, effective code out there in these languages.

Or perhaps you are confused about the functional programming part. This is only strongly associated with Scheme as other Lisps support other paradigms like object-oriented programming. Functional programming is a thing that takes a while to understand, although my theory is it's actually more natural and it's only because you already learnt imperative programming, which is thoroughly unnatural, that you find it odd. Once grasped it will help you with other languages that support functional programming like JavaScript and Python.

They say learning Lisp will make you a better programmer even if you never use it again. I tend to agree with this.

[+] mattparas|2 years ago|reply
At least for me, there is a certain appeal in building the world. With scheme you get a very small set of functionality, but you use that to implement the rest of the language, and build abstractions on top of abstractions. Seeing how the whole system can be built from a small set of functionality is pretty cool, and also very satisfying. There is a talk from Andy Wingo, one of the maintainers of Guile, where he describes working on Guile as akin to tending to a garden, and I think its an apt comparison. Something about it feels very organic and personal, which is part of the appeal.

The syntax itself doesn't _really_ matter, it just makes it easy to do so - functions and syntax visually look the same, so it makes it easy to build.

Its not for everyone, but I think its worth exploring for a little bit. Similarly I think its worth really learning any language just a bit, if not to just expand your tool kit. The parenthesis do disappear at a certain point and you learn to read it, but if its not your thing that is fine.

[+] femiagbabiaka|2 years ago|reply
Basically the differences are in the concepts you'll use to write code. Lisps themselves are very different from each other, but just like the languages you're used to, many lisp distributions have standard libraries that can be called, and those building blocks can be used to build applications or whatever else. In this case specifically, Steel provides the facility to call Rust functions within a Steel program: https://github.com/mattwparas/steel.

So, although I haven't used Steel, it looks like the advantage you'd get from using it is the opportunity to take advantage of features it provides like transducers and contracts, which are feature common to some other Lisps as well.

So, just like choosing any other language, it boils down to a series of tradeoffs.

[+] dboreham|2 years ago|reply
You're not missing anything. You have two choices: a) either take it as true that there's nothing to see here and move on, b) or if you have some spare time learn lisp and then move on. (b) avoids that nagging feeling that you're somehow unworthy.
[+] tmtvl|2 years ago|reply
My first introduction to Lisp-like programming languages were the SICP videos with Abelson and Sussman so to me Lisps are just programming languages like any other but with a nicer syntax and great support for interactive programming.
[+] dkarras|2 years ago|reply
used to be on the same camp. now I think I get it.

the language is malleable. because it is homoiconic. so while developing software, you are simultaneously writing a domain specific language for your problem. because macros.

in the end, if you like your craft, you end up with a "language" that is very suitable for solving the problem you have at hand, with very little noise.

the downside is, probably most others will not understand your code so lisps have heavy bias towards "solo hackers". BUT... if that is important to you, you can code towards understandability. so much so that you can make it very hard for others to make mistakes when using the public api.

so with most programming languages, you program within the constraints of the syntactic rules of the language. with lisps, you define the language you want to approach the problem, that language comes out by itself iteratively. for some, that is a joy. others don't care for it.

[+] darthrupert|2 years ago|reply
This is just perfection on first glance, exactly what I've been looking for to augment my Rust projects that need a command/configuration language. And with a working repl, too, apparently.

I hope the second and third glances will be good too.

[+] CooCooCaCha|2 years ago|reply
How does Steel handle garbage collection? Would it be possible to manually control the garbage collector? For example, a game that runs the GC at the end of every frame.
[+] mattparas|2 years ago|reply
Immutable values are reference counted, so for most code, things will be dropped when they exit scope. For captured mutable values, there is a fairly mundane mark and sweep collector. It is possible to manually control the garbage collector, however I have not optimized it for that kind of workload. If you were embedding Steel in a game, I don't think it would be explicitly necessary to tune the GC as long as you aren't using a ton of mutation. If you were using a lot of mutation and still wanted a relatively performant GC collection at the end of every frame, then the underlying GC implementation would have to be changed or swapped to a different one (which is not impossible - I just only have one GC implemented)
[+] nerdponx|2 years ago|reply
Which Scheme is this implementing? R7RS? Any SRFIs? Is there a library reference document?
[+] mattparas|2 years ago|reply
Starting with R5RS compliance, then once that is achieved moving on to R7RS.

I borrowed the R5RS test suite from chibi here - https://github.com/mattwparas/steel/blob/master/cogs/r5rs.sc... - only a few of these tests don't yet pass. Something like 135 pass, 4 fail, 20 skipped since I haven't implemented the primitives yet.

I've only tested against a few SRFIs so far, but am also attempting to run the R7RS benchmark suite https://ecraven.github.io/r7rs-benchmarks/ - So far you can view the progress here https://github.com/mattwparas/steel/tree/master/r7rs-benchma...

There are some more that aren't yet checked in. I plan to get a document up with the exact state of compliance soon.

The biggest difference right now is that, like Racket, Steel lists are immutable, so there I need a compatibility layer when running portable scheme.

[+] perrygeo|2 years ago|reply
Fantastic project! If you're looking to embed a DSL/full programming language into your application, this seems right on point. Rhai and RustPython are two other options if you don't dig parentheses.
[+] tucnak|2 years ago|reply
Is it possible to avoid rust completely when programming using Steel?
[+] mattparas|2 years ago|reply
Yep - you can write standalone steel code without interacting with Rust at all, just interacting with the interpreter. I've done the first few days of the advent of code in Steel without needing to touch any Rust. Now, I will say that Steel has gotten visibility faster than I've been able to keep up with, so you might find a native function missing, or something that you would like to use that isn't implemented yet - at which point you either need to implement it yourself or open up an issue for someone to get to :)
[+] silent_cal|2 years ago|reply
Maybe a dumb question but does embeddable mean embeddable in web browsers? If so it would be awesome to include this in a web-based version of SICP.
[+] I_am_tiberius|2 years ago|reply
It sounds great but what is it exactly and what can I use it for? Is there an equivalent of this in other languages?
[+] mattparas|2 years ago|reply
Hopefully the linked README provides a general overview (I know I need to write some more documentation!), but Steel is an implementation of the scheme programming language (not entirely compliant yet, but aiming for R5RS and R7RS compliance). It can be used as a standalone language via the interpreter/repl (like Python or Racket), or it can be embedded inside applications, like Lua. There are hundreds (thousands, probably) of embeddable languages, each with their own flavor - see a list compiled here for example https://github.com/dbohdan/embedded-scripting-languages

Use cases are generally for either configuration, scripting, or plugins - so scripting in games, or adding extensions to your text editor without having to use FFI or RPC + serializing a bunch of data. The advantage it has over using dynamic libraries (in general) is it runs in the same process, and can access the internal data structures directly without a lot of ceremony involved. The downside is that it is typically not as fast as native code unless a JIT is involved.

Javascript is an example of an embedded scripting, where the browser is the host application.

[+] sirsuki|2 years ago|reply
When I tried to introduce s-expressions to a DSL my co-workers nearly lynched me. The parenthesis were so violently hated I sunk into a deep hole and still haven’t came back out of it.
[+] mtlmtlmtlmtl|2 years ago|reply
It really is kind of frustrating isn't it? Because anyone who's put any real effort into learning Lisp knows that the parentheses are not what's hard to understand, actually. You don't even have to read them, really.

So when people complain loudly about parentheses in Lisp that just tells me they probably never made any real effort to learn it, and are actively opposed to trying.

It's not a productive starting point for a discussion.

[+] brundolf|2 years ago|reply
I did it once in an internal UI because we needed to rapidly expose some functionality that could be composed/piped in complex ways and it would take a while to implement a normal UI

Was met with skepticism, esp around the engineering/maintenance overhead, until I told them the parser took me a couple hours to write and was only a hundred lines of code or so

[+] capableweb|2 years ago|reply
It is really fun (sad) to see this, I've seen it so many times myself too. You show how something would be with s-expressions, compared to something, and all they can focus on is how many parenthesis there are. But when you sit down and count, they're the same amount as the code was when it wasn't s-expressions, just in different locations. And when you remove the parenthesis, they can kind of understand the code, kind of.
[+] darthrupert|2 years ago|reply
The things I'm looking for in language syntax these days are

1. That it's unambiguous enough that my editor can format it correctly every time (provided the code is correct, obviously). Indentation-based languages like Python fail this.

2. That its elements and keywords are distinct enough that my editor can apply colours on it. I sometimes feel that S-expression languages fail at this because of their small number of distinct keywords, but that might not be true.

Besides these two points, if the language is not a joke language, it's probably fine.

[+] oldpersonintx|2 years ago|reply
I want this to succeed!! please do not let the word "srfi" ever appear in the packages list...naming libraries with obscure numbers no one remembers was a terrible terrible idea that all schemes seem to perpetuate
[+] aag|2 years ago|reply
SRFI editor here. The numeric designations are there because, among other reasons, there is sometimes more than one SRFI for a particular general idea. But there is nothing stopping libraries from having more than one name, including a semantically meaningful one. In fact, there has been a SRFI standardizing how that is done since 2008 [1].

[1] https://srfi.schemers.org/srfi-97/srfi-97.html

[+] zelphirkalt|2 years ago|reply
It is a standard. What do you expect? But there are some or many Schemes that offer their own libs on top or below SRFI implementations, which then have readable names.
[+] mattparas|2 years ago|reply
At the moment I only have 1 SRFI package, and its just to check against compatibility :) - my plan is to wrap the SRFI packages with friendly names and just include the metadata in the package spec so someone searching for it can find it easily.
[+] wolverine876|2 years ago|reply
I wonder where the name came from. Being HN, here is my nitpicking imperative:

Names are important. Our inheritance is wit, self-awareness, and irony; names that puncture ego and power and that appeal to joy: C, C++, GNU, Rust, Google, Yahoo!, Vim, Git, awk, etc. Others are beautiful, evocative images, like Apple and Amazon. Names communicate our culture and ideals to each other and to the next generation.

Careless, thoughtless names like Microsoft, IBM, etc. (ok it's ironic and self-deprecating, but without self-awareness or wit!), etc. should be hated and banned. Egotistical BS, especially Tolkien plagiarists who assert they are supernatural, should be tarred and feathered and paraded around town (with wit and irony).

(Plenty of names fall in some middle ground.)

If Steel is just a derivative of 'Rust' [edit: it is not, see the response below], it misses the self-awareness. Someone naming their development product - designed to build great structures - 'Rust' is engaging in a little self-deprication, joy, and self-awareness. Naming the derivative project 'Steel' possibly misses all that; there's a reason the original wasn't named Iron or Steel or Carbon Fiber. But maybe there's more to the name.