[evadne]

Regarding numeric stuff. I discussed this matter with some folks and it was quite interesting because just like Erlang/OTP was used at control plane for phone switches while the data plane was implemented in something else, one could theoretically leverage it to oversee ML processes (perhaps Python programs written with PyTorch or something else)

The topology is exactly the same

In fact, porting your vetted numeric code to Elixir is probably going to make the risk go sky high, but running the same code with supervision structure in Elixir would in fact reduce risk so is a no-brainer

Just my thoughts of course

[ricketycricket]

I do exactly this. Elixir supervises Ports to python and C binaries and manages all the communication between processes including buffering and maintaining state when necessary. Using libraries like Horde even allows you to run these processes on remote nodes very easily. It has worked fantastically well.

[datadeft]

Elixir and Explorer using the Rust implementation of Polars under the hood is practically the same as Python and the Polars Python bindings. The rest of the programming language makes a difference though. Python rejected a large portion of CS research and rejected to do a proper implementation for many things like pattern matching or lambda functions. I am pretty sure they had sound reasons to do this.