How to rewrite it in Rust

[sorenbs]

We did a similar thing with a Scala -> Rust rewrite for the http://prisma.io query engine.

By rewriting small components and integrating them into the existing project using Javas native interface, our small team of 5 developers were able to pull off this massive rewrite in just under a year. The resulting code base is rearchitected in a few very important ways, but mostly follows the same structure.

And because we kept and evolved our old Scala based test suite, we have a very high confidence in the rewrite.

When Async/.await finally landed, we could switch over very quickly, and it has been a joy to focus on benchmarks and performance over the last month. Spoiler: Rust is faster than Scala :-D

[tombert]

I promise that this is asked genuinely and isn't some sort of veiled "gotcha!" (it's tough to tell on the internet sometimes); what was the reason for a change from Scala to Rust?

I ask because Scala already has a good type system and the JVM typically has good performance nowadays, particularly with something like GraalVM, so I am actually really curious to why you felt a Rust rewrite was a good idea.

[sorenbs]

I should add that the Rust community has been extraordinarily welcoming, and our existing Scala engineers were able to relatively quickly become proficient in Rust.

Huge shoutout to everyone working on Rust Async, especially the Berlin crew, who has been very helpful.

[yann_s]

It'd be great to hear from your experience about using Javas native interface. AFAIK, using native call can have a performance impact: https://en.wikipedia.org/wiki/Java_Native_Interface#Performa...

I guess the plan what to make native is pretty important. Maybe you want to disclose more details about that?

[michael_j_ward]

Do you have any write-ups on this? I might be looking to do this for a large java codebase soon.

[Scarbutt]

How many lines of code? (old/new)

[pornel]

This ability to incrementally add Rust to a C codebase is very useful for adopting Rust in established projects.

You don't actually have to rewrite everything on day one. You can stop writing new C code right now, and then gradually replace old code only when you need to fix it or refactor it.

[Sammi]

Twice I've been part of a move from Javascript to Typescript that worked much the same. Both projects were large applications with several developers working on them. Both had been ongoing for a few years before the port started. In both projects we decided to write all new code in Typescript and convert JS to TS when we made any largish change to an existing JS file. In both cases it took around a year for us to hit > 90% all code being converted this way, and at that time we decided to actually make issues in our issue tracker for porting the rest, and then had the rest converted in a couple of months after that.

The big difference is however that JS and TS can live side by side on a file by file basis out of the box with the Typescript compiler, which makes it super easy to convert. You don't have this luxury with C and Rust of of the box, but serious kudos to the author for finding a way to do something very similar.

When converting C to Rust you usually have to do things on a module by module or compile artifact by compile artifact basis, which makes it much more challenging. You can however employ some sort of strangler pattern: https://docs.microsoft.com/en-us/azure/architecture/patterns...

[dfox]

Is there any way to do this the other way around and integrate new rust code into non-trivial C project with non-trivial build system?

[saagarjha]

> However, Rust has a killer feature when it comes to this sort of thing. It can call into C code with no overhead (i.e. the runtime doesn’t need to inject automatic marshalling like C#’s P/Invoke) and it can expose functions which can be consumed by C just like any other C function.

As we see below, you may still need to write some code to convert from C types to something that is more ergonomic to use in Rust. But the marshaling ABI-wise is minimal.

> Turns out the original tinyvm will crash for some reason when you have multiple layers of includes

It's actually crashing because there are no lines of code in the file, so certain data structures never get initialized.

[rvz]

> As we see below, you may still need to write some code to convert from C types to something that is more ergonomic to use in Rust. But the marshaling ABI-wise is minimal.

Exactly, as much as I love writing Rust code, there is nothing that is more frustrating that maintaining bindings from C to Rust. Then you'd have to create another idiomatic binding on top of it. Sure bindgen is great, but Zig's cImport and Swift's ClangImporter take this further.

They both use clang modules to tackle the first part. A autogenerated idiomatic solution would be great for Rust, but sadly it doesn't exist.

[ridiculous_fish]

> This is where build scripts come in. Our strategy will be for the Rust crate to use a build.rs build script and the cc crate to invoke the equivalent commands to our make invocation

Yikes - port the entire build system to cargo before you write a line of Rust. Now draw the reset of the owl!

Surely's there's an incremental path for the build as well? Perhaps if you're using CMake?

[pornel]

You can build Rust code as a static library and make the C build system consume that instead. This is the approach that librsvg took.

Moving C build to build.rs is not necessary. It's usually done only because people used to Cargo don't like bringing CMake along. If you were to publish this as a Rust crate, it'd be slightly easier for downstream Rust users to have one less external tool to install.

[Rusky]

There's a `cmake` crate that you could use from `build.rs` the same way, or you could go the other direction and have your existing build system invoke Cargo or rustc.

[staticassertion]

Great guide, this looked surprisingly straightforward.

[wheaties]

I came here to say the same thing. This is a great article on "Rust is safe for our org."

[bfrog]

This seems exactly what remacs is doing with emacs.

[fortran77]

What language will it be _next_ year?