Roto: A Compiled Scripting Language for Rust

[airstrike]

> Roto fills this niche for us. In short, it's a statically typed, JIT compiled, hot-reloadable, embedded scripting language. To get good performance, Roto scripts are compiled to machine code at runtime with the cranelift compiler backend.

Statically typed, hot reloadable scripting? Sign me up.

[echelon]

I have dreamed of this for Rust. It is literally _the_ killer app to have a fast scripting language that pairs with Rust well.

With fast development cycles and a safe scripting language, Rust will find itself in every single programming niche. Server development, game development, desktop application development. If the WASM DOM bridge gets better, maybe even web development. Everything will open up.

You can prototype in the scripting language, write mature application logic in the scripting language, and for the parts that matter, move them to Rust.

I hope Roto can be that. There are lots of other Rust scripting languages that fall short.

A good scripting language will have great Rust interop, similar syntax, and allow simple calling and passing of data structures. It shouldn't weaken Rust in any way.

[speed_spread]

Then also have a look at Mun:

https://github.com/mun-lang/mun

- Ahead of time compilation

- Statically typed

- First class hot-reloading

Not sure how both languages compare though

[ijustlovemath]

In case the author comes in, I'm curious about how you designed the registration mechanism. We have a Python application that makes heavy use of decorators to give strong runtime introspection capabilities, and I've always wondered if the same could be done at or near compile time in an equivalent Rust context. I think learning about the drawbacks and boons of the designs you settled on would be really informative!

[terts]

Hi! So for Roto, our introspection needs are actually fairly limited. We only need the `TypeId`, the type name, the size and the alignment. which Rust can give us without any additional traits. It's not possible currently to - for example - access struct fields and enum variants. That is something that I plan to add, but that might require a crate like `bevy_reflect` or `facet`.

Rust is giving me just enough information to pull the current version of. More powerful introspection/reflection is not possible without derive macros. If you're ok with derive macros though, you could look into the 2 crates I mentioned.

Hope that answers your question!

[dkarl]

Could you write 80-100% of an application in this language? I'm wondering if it could be a good application language for Rust programmers who want to use the Rust ecosystem and have the option of writing parts of their application in Rust for extra performance, but who also want to experiment and iterate quickly, and who want a simpler, higher-level language for expressing business logic.

[terts]

Hi! Author here. You could try but there are some fundamental limitations.

The biggest limitation is that we don't have access to the full type system of Rust. I don't think we can ever support registering generic types (e.g. you can register `Vec<u32>` but not `Vec<T>`) and you don't have access to traits. So it would work if you can reduce your API to concrete types.

Otherwise - apart from some missing features - you could probably write big chunks in Roto if you wanted to. You could also prototype things in Roto and then translate to Rust with some simplified types.

Also you'd have to accept that it gets compiled at runtime.

[0cf8612b2e1e]

That’s my dream. Seamless FFI to Rust with all of the core in a dynamic scripting language. If/when you need to tighten up performance/types, can port more of the code to actual Rust.

[ori_b]

This language looks a lot like Rust. Why not dlopen() a Rust shared library instead? The implementation would be about as complicated, but it would be a well known language that's fully integrated with a large library ecosystem, well defined build and package set, rather than some custom one-off thing with no ecosystem. Going your own way means your users have to re-invent the wheel for themselves every time.

With Rust's safety, it's not even that bad to re-open and re-load the binary when it changes; the big footgun with dlopen is use-after-free.

[terts]

Hi! Author here. There's a couple of reasons.

First, this language is syntactically a lot like Rust but semantically quite different. It has no references for example. We're trying to keep it simpler than Rust for our users.

Second, using Rust would require compiling the script with a Rust compiler, which you'd then have to install alongside the application. Roto can be fully compiled by the host application.

I think your approach might be preferred when the application author is also the script author. For example, if you're a game developer who is using a script to quickly prototype a game.

[abendstolz]

Don't rust shared libraries have the problem of no stable rust ABI? So you either use the C ABI or you use some crate to create a stable rust ABI, because otherwise a shared lib compiled with rust compiler 1.x.y on system a isn't guaranteed to work with the binary compiled on the same system with another compiler... or on another system with the same compiler version.

Right?

[abendstolz]

Very cool!

But I prefer the wasmtime webassembly component model approach these days.

Built a plugin system with that, which has one major upside in my book:

No stringly function invocation.

Instead of run_function("my-function-with-typo") I have a instantiated_plugin.my_function call, where I can be sure that if the plugin has been instantiated, it does have that function.

[bobajeff]

This sounds like a good approach to overcoming rusts slow compile times and lack of dynamic linking. One thing I'm concerned about with this path is what about hot reloading and fast script running? Doesn't everything in the wasm component model need to be compiled first? I imagine that would remove some of the advantages to using a scripting language like JavaScript or Python.

[zamalek]

Alternatively, whenever designing a scripting/plugin host make sure to support plugin-hosting-plugins. That way you could have the Roto plugin host complied to wasm.

[90s_dev]

How is that not also stringly typed?

[90s_dev]

> Finally, we want a language that is easy to pick up; it should feel like a statically typed version of scripting languages you're used to.

It looks like Rust. All Rust scripting languages do. Is this true for all other languages? Is this just a property of embeddable scripting languages, they will always resemble the language they're implemented in and meant to be embedded in?

[Philpax]

People who become proficient in Rust generally enjoy the syntax, so they want to carry it across. (As someone proficient in Rust who has pondered their ideal scripting language, I would have done the same.)

To your more general question: it depends. AngelScript [0] looks very much like C++, while others, like Lua, don't. It's really up to the designer's discretion.

[0]: https://www.angelcode.com/angelscript/, but https://angelscript.hazelight.se/ has better examples of what it actually looks like in use

[epage]

> It looks like Rust. All Rust scripting languages do.

Not koto (https://koto.dev/) which is one of the reasons I appreciate it. I want an embeddable language targeted at my users, rather than myself which I feel Rust-like ones do. I also want an embeddable language not tied to my implementation language so if I change one, I don't have to change both. Koto only supports Rust atm but I don't see why it couldn't be supported elsewhere.

[nicoburns]

I think it's just that a lot of people like Rust syntax, and there is a lot of demand for a Rust-like scripting language (Rust syntax is also very close to JavaScript/TypeScript syntax which many, many people are familiar with)

[andsoitis]

> Is this just a property of embeddable scripting languages, they will always resemble the language they're implemented in and meant to be embedded in?

No. Think about Lua or Tcl (both implemented in C) or others like Embeddable Common Lisp.

[pansa2]

IIRC Lua deliberately doesn’t resemble C - so if you’re going back-and-forth, editing both the host application code and a script, you can immediately tell which one you’re looking at.

Makes sense to me - which means scripting languages for curly-brace languages should probably use either Lua-style begin-end or Python-style significant indentation.

[breadchris]

I like yaegi [1] for go because it is an interpreter for the go language (almost fully supported, generics need some love). The most important part for me is being able to keep all my language tooling when switching between interpreted/compiled code. Also, there is little needed distinction between what is going to be interpreted and compiled. Once you start including libraries it gets dicey and the need for including the libraries in the compiled part is necessary. There is also a blog post that comes along with it describing how it was built! [2]

[1] https://github.com/traefik/yaegi [2] https://marc.vertes.org/yaegi-internals/

[lynndotpy]

Woah, this looks awesome.

One of my favorite things about writing Rust is that it's expression-oriented (i.e. almost everything is an expression), something you almost never see in non-functional languages.

I was wondering if Roto is also expression-oriented?

[terts]

Hi! Author here. It is indeed expression-oriented, mostly following the same rules as Rust. If-else is an expression, for example.

[90s_dev]

> Note that nothing in the script is run automatically when the script is loaded, as happens in many other scripting language. The host application decides which functions and filtermaps it extracts from the script and when to run them.

So it's closer to something like C or C++, where it just defined stuff and you can choose what to use? I guess that's fine when there's no initialization for the script to do. Maybe in your domain that's never the case. But many languages end up adding static initialization as a first-class feature eventually.

[stirfish]

Roto means "broken" in Spanish.

[diggan]

Just add a "Huevos" prefix and a "S" at the end of the name, and suddenly you're thinking of delicious food instead.

[hannofcart]

> Roto has no facilities to create loops. The reason for this is that scripts need to run only for a short time and should not slow down the application. [1]

Wait, what?! Isn't that choice a bit extreme?

There have been plenty of times in scripting where I've needed loops! Am I missing something here?

[1] https://rotonda.docs.nlnetlabs.nl/en/stable/roto/00_introduc...

[terts]

Hi! That bit of the docs is a bit outdated. We're probably gonna make it optional. The reason for that choice was that the filters for Rotonda need to be very quick and don't really require loops as long as you can do `contains` checks on some lists for example.

So it should probably say "Roto _in Rotonda_ does not have loops". Roto the language as a separate project can then have loops.

[lewisjoe]

ELI5: Why not use typescript and an embedded v8 engine or deno to run them? What kind of advantages will I miss if I go for typescript as an embedded language for my application?

Also by using v8, I open up new possiblities via wasm (write in any lang and run it here?)

Will be helpful if somebody enlighten me.

[duped]

Be warned that V8 is a behemoth and adds 100+MB to your binary size, is quite slow to link, and is not practical to compile from source for most projects. If you want a lighter weight JS runtime there's quickjs, if you want WASM there's wasmtime.

Personally I don't think it's a good choice for what it seems Roto is used for (mission critical code that never crashes) since TS doesn't have a sound typesystem and it's still very easy to fuck up without additional tooling around it.

[skybrian]

Roto is a very limited scripting language with no loops. You might compare it with the eBPF language used to load filters into the Linux kernel.

[giancarlostoro]

You are comparing a general purpose scripting language (TypeScript) to a DSL (Domain Specific Language) essentially. They built theirs with a specific purpose.

[airstrike]

Why would I want to bundle an entire JS runtime? And why do you think you need that for WASM?

And personally I will go out of my way to not use TS/JS if I can

[ijustlovemath]

sounds like it's not fast enough for their use case. plus, have you ever tried to integrate v8 into a project? Deno is fine for building binaries, but to date doesn't really have good support out of the box for bundling a script into a library, which this application seems like it would need.

[ost-ing]

Any chance of no_std support for embedded systems?

[terts]

Hi! Author here. Would be super cool, but while the compiled scripts would work without allocations, the compiler itself does a lot of allocations. So unfortunately I don't think I could make that work.

[samuell]

Sidenote, but I wish it became the practice to explain abbreviations such as BGP, to make the post intelligible to people outside the field.

[jpc0]

Border Gateway Protocol

But I don’t even think people in networking would say that, it is canonically BGP.

This is kind of like complaining about the abbreviation HTML, sure yes it is Hypertext Markup Language but everyone knows it as HTML to the point that there are probably people that don’t know it’s an abbreviation.

[gaugefield]

There is a convention in academic papers where you write the explanation for the 1st occurrence of the abbreviation, then leave it out for the rest of the paper. I suggest others to follow the same, except maybe for the most obvious ones (like HTML mentioned in the other reply)

[belter]

Its a short blog about routing rules in a networking domain by NLNet Labs. The explanation of the BGP abbreviation is uncalled for, just because Rust tourists were attracted to click on it, the moment they saw the word...

[ape4]

Especially TLAs

https://en.wikipedia.org/wiki/BGP_(disambiguation)

[terts]

Hi! Author here. Yep, sorry about that. It is indeed Border Gateway Protocol.