(no title)
sd2k | 1 month ago
I've been working on a couple of things which take a very similar approach, with what seem to be some different tradeoffs:
- eryx [1], which uses a WASI build of CPython to provide a true Python sandbox (similar to componentize-py but supports some form of 'dynamic linking' with either pure Python packages or WASI-compiled native wheels) - conch [2], which embeds the `brush` Rust reimplementation of Bash to provide a similar bash sandbox. This is where I've been struggling with figuring out the best way to do subcommands, right now they just have to be rewritten and compiled in but I'd like to find a way to dynamically link them in similar to the Python package approach...
One other note, WASI's VFS support has been great, I just wish there was more progress on `wasi-tls`, it's tricky to get network access working otherwise...
[1] https://github.com/eryx-org/eryx [2] https://github.com/sd2k/conch
souvik1997|1 month ago
For `jq` specifically we use the excellent `jaq_interpret` crate: https://crates.io/crates/jaq-interpret
curl is interesting. We don't include it currently but we could do it without too much additional effort.
Networking isn't done within the wasm sandbox; we "yield" back to the the caller using what we call "host operations" in order to perform any IO. This keeps the Wasm sandbox minimal and as close to "pure compute" as possible. In fact, the only capabilities we give the WASI runtime is a method to get the current time and to generate random numbers. Since we intercept all external IO, random number generation, time, and the Wasm runtime is just for pure computation, we also get perfect reproducibility. We can replay anything within the sandbox exactly.
Your approach with brush is interesting. Having actual bash semantics rather than "bash-like" is a real advantage for complex scripts. The dynamic linking problem for subcommands is a tough one; have you looked at WASI components for this? Feels like that's where it'll eventually land but the tooling isn't there yet.
Will check out eryx and conch. Thanks for sharing!
sd2k|1 month ago
I did something very similar to you for networking in eryx too (no networking in conch yet); defined an `eryx:net` interface in WIT and reimplemented the `urllib` module using host networking, which most downstream packages (httpx, requests, etc) use far enough down the stack. It's a tradeoff but I think it's pretty much good enough for most use cases like this, and gives the host full control which is great.
Oh full transparency, the vast majority of conch and eryx were written by Opus 4.5. Being backed by wasmtime and the rather strict Rust compiler is definitely a boon here!