(no title)
pchiusano | 2 years ago
There's some interesting subtleties with FFI in a distributed programming language, so I'll ramble about that here in case it's interesting to you. :)
So, in Unison, all values are serializable, including functions and their dependencies. This is a key superpower that enables a lot of the neat stuff we do. As long as those functions are written in pure Unison, we can easily serialize them, deploy them on the fly, etc. But when we add an FFI, the story changes - the sender may have some C library in their environment, and that library may not exist at the recipient node. The two nodes could be different platforms, one Mac, one Linux, and the library may have been written specifically for Linux, say!
So when we add FFI, we will likely be doing it in a different way than most languages. Functions that use FFI will have this tracked in the their type, using our effect system. You'll be able to use whatever C libraries you want in your local computations, but if you want to sent those values around, you need to be sending them to a place that supports that same set of FFI effects, since the C library and its dependencies can't literally be deployed on the fly in the same way as pure Unison code. In our cloud platform, nodes are typed based on what effects they support and we'll probably add a way to create new node pools that have access to whatever C libraries you want.
In "regular" programming, we're not used to thinking about "the execution environment" as a thing that's represented explicitly within the program. Instead, there's an assumed execution environment (which includes the set of native libraries, etc) and you get runtime errors if you run a program and some of the assumed execution environment is missing (like a shared library, say). For the most part, people have been okay with this, but it's already somewhat of a problem for languages that target the browser and the backend. The language may be statically typed, but now the type system is not tracking some key information - namely, is this a function I can call here (if I'm expecting this code to compile to JS) or is it a function that only can be called for backend code? In a distributed setting with heterogeneous nodes with different capabilities, this problem is even more pronounced, which is why we track this information in the types and plan to do so once we add FFI.
Hope that was interesting!
[1]: https://www.unison-lang.org/blog/jit-announce/ is an early progress report, and I think we're finally shipping something in the next month!
59nadir|2 years ago
It's definitely interesting. I'm happy to hear that you'll definitely be supporting what we can currently do with less rigid languages and their FFI, but that we'll have to adapt to the bounds of the system and its rules to some extent.
Do you have any idea how you'd support `LoadLibrary`/`dlopen` in order to load/reload function pointers live? At some point I imagine a lot of this has to have a "trust me" kind of escape hatch.