CPython's main branch running in the browser with WebAssembly

[ridiculous_fish]

One of the painful aspects of WASM is there's no blocking calls. You can't say "wait for the next event;" instead you must return to the outermost event loop, and wait to be called back.

How does Python-in-WASM work around that? For example, how does `for line in sys.stdin:` work if you can't actually block on stdin?

Emscripten has some support for this via the "asyncify" transform, which layers additional control flow to enable return all the way up the call stack, and then "rewind" back down into it. But this bloats the code (and is also buggy) so maybe it's not being used.

[ethanhsmith]

Yes, currently input goes into a propmpt() and it doesn't output anything unless you hit "Cancel" on the prompt, definitely a bad time.

Python allows you to reach in and replace the core interpreter loop, so this may be an avenue to have our own asyncify-like function pop out to JS land and restore state correctly (which we can be smart about since we are the interpreter).

It may also be possible to write something that runs Python in a webworker and communicate with it over a sharedarraybuffer, but that I'm a bit more hazy on. Pyodide has some discussion of this in https://github.com/pyodide/pyodide/issues/1219 and https://github.com/pyodide/pyodide/issues/1503.

This is definitely the hardest part of getting Python to work. Well, hardest after the hardest part of building a compiler toolchain like Emscripten :)

[bkolobara]

It's on the embedder (Wasm VM) to provide this functionality. I'm working on a Wasm runtime [0] that is written in Rust and uses stack switching to allow you to call Rust async functions as if they were blocking. This keeps the Wasm bytecode simple (blocking), but at the same time provides high performance i/o.

There is also a proposal to bring stack switching to the browser.

[0]: https://github.com/lunatic-solutions/lunatic

[samwillis]

I know Absurd SQL[0] uses SharedArrayBuffer and Atomics to turn the async IndexDB into sync for use by Wasm. I wander if it’s possible to use that here too although it’s obviously a little different?

0: https://github.com/jlongster/absurd-sql

1: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

2: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

[bayesian_horse]

For Pyodide I made a proof of concept where you can run async/await with promises.

If people actually develop stuff in Python for the web, they should do something like that.

[trothamel]

In my experience, asyncify works pretty well everywhere but Safari on macOS/ARM or ios/ARM, where the stack sizes are too small to be useful. You do want to be a bit careful about where you block, which can minimize the number of functions that need to be transformed.

[azakai]

> and is also buggy

If you've found any bugs in Asyncify please file them! There are no open issues atm about any general bugs, aside from some corner cases with features like dynamic linking.

[lights0123]

You can use Atomics.wait() to block a non-main thread.

[trothamel]

Although it's only Python 2.7, cpython-based games have been running in the web browser for years.

https://beuc.itch.io/the-question-web

(I'm the lead developer of Ren'Py, though Sylvain Beucler did most of the work. He also has a 3.8 port here: https://www.beuc.net/python-emscripten/python/dir?ci=tip )

[dmw_ng]

The PyPy team demoed a networked multiplayer browser-compiled Python game at EuroPython in 2006. Anyone remember the details? Seems it has dropped off the face of google.

[doublepg23]

Ren'Py is a joy to use and an excellent project.

[pwang]

The homepage that's linked from Github seems to be down? https://renpy.beuc.net/

[ethanhsmith]

Yep! I definitely want to build on and learn from existing patched versions of Python running in the web. Do you know what you folks do for synchronous I/O calls?

[mrich]

It's a pity WASM and all the tech around it wasn't available 20 years ago when Javascript was added to the browser, we'd all be using python now and node.js wouldn't exist at all ;)

[tgv]

It would be just like Javascript, only slower.

[easton]

Guido responded and wondered if this could be integrated into github.dev for Python work in the browser without remote compute. That’s a very cool idea, but I wonder if this would work any better with the usual suspects (pandas, numpy, mathplotlib) than the other attempts like pyodide which make more modifications to CPython.

[WithinReason]

What is the issue with Pyodide?

[petters]

What is the novelty here?

E.g. we have https://github.com/pyodide/pyodide and other examples.

The cool part here is Emscripten, which has been around for a long time.

[simonw]

It's pretty cool that this latest version uses the latest main branch of CPython directly, without any additional modifications.

[pjmlp]

And before it we had Python on the browser via ActiveX and ActiveState's plugin.

[pansa2]

I imagine the biggest downside of this approach is simply the size of the CPython implementation. Does anyone know how big it is when compiled to WASM?

I wonder if anyone has tried the same approach using MicroPython.

[benno128]

There's a version of CPython 3.6 compiled here (https://wapm.io/package/python) which is ~5mb gzipped.

[wheelerof4te]

This is it. This is the beginning of a revolution. Prepare your fork-picks.

Jokes aside, JS ecosystem really needs a competitor. Web developers have been cutting corner after corner for decades, with ever increasing disregard for performance and memory consumption.

Now with both Python and Rust in the browser, things may change for the better.

[paavohtl]

CPython is orders of magnitude slower than modern JavaScript engines. Especially so when the interpreter is compiled into WASM.

It is interesting from an interoperability point of view, but this purely negative from a performance and maintainability point of view.

[bredren]

I’ve been primarily a Python web dev, focused on Django.

But in picking up modern frontend, even in the past two years, the frameworks and toolchains have matured or simplified a great deal.

The training options are plentiful. To some extent, Node has Deno “competing” to add features and provide performant backend.

Esbuild, for example, is built on rust and unlocked massive performance improvements in bundling.

It seems like JS is in a better position than it has ever been.

[kello]

I really want you to be right, but what incentives does your run-of-the-miil web dev hav for switching to WASM + (pick language here)?

[brian_herman]

https://www.destroyallsoftware.com/talks/the-birth-and-death... Just a matter of time now.

[infocollector]

This looks exciting. If anyone knows how to, perhaps they can report how much space this uses in the browser? Some benchmarks? Someday soon I hope we can use this for developing code on the browser to aid with web applications.

[ethanhsmith]

Hi, author of the tweet here

It is definitely too early for benchmarks, this is a "I got it working!" update.

The original data file with all of the standard library was a bit over 200MB. Slashing what isn't going to be run in the browser (e.g. tkinter) and zipping the standard library got it down to about 20MB. There is probably more that could be removed, and there are modules we don't need to build that we currently do. There are other things we can do like set the less frequently used modules to be loaded asynchronously.

While I doubt this will be production ready "soon", I do hope to keep working on fixing bugs and such.

[stevefan1999]

How does that compares to Brython [1]?

[1]: https://brython.info/

[ethanhsmith]

Brython is a complete re-implementation of Python, thus it doesn't support some features/libraries (at least, it didn't when I tried it last), and is not compatible with C extensions.

The demo I put in the tweet is the same code as when you type `python3` in the terminal, just running in the browser. So it is much more compatible and is mostly [1] feature complete.

[1] minus whatever libraries are likely never to be used that we ripped out

[pansa2]

See "Running Python in the Browser" [0] - I believe this approach is similar to Pyodide's.

[0] https://yasoob.me/2019/05/22/running-python-in-the-browser/

[matiasb]

Very cool, I did some experiments with libxml on a WASI environment a few years ago (https://github.com/matiasinsaurralde/wasm-libxml2).

[qqumut]

This is like reverse cloud system. Don't run on cloud, or in your os, run on your browser.

[wheelerof4te]

So, Castle in the sky system?

[vermilingua]

How is it directly replacing the JS REPL in the console? I didn’t know that was possible.

[teaearlgraycold]

It's not. The output appears in the console but input is provided in the page.

[tspike]

Is there DOM access?

[999900000999]

Would it be possible to use Python directly in say Chrome, assuming someone pushed a PR for chromium to do this?

[jagger27]

The beauty of WebAssembly is that you don't need Google's permission to add support. Just send your Wasm blob to the browser and Chrome's existing Wasm runtime will just run it.

[leventt]

There is this: https://pyodide.org/en/stable/console.html Not sure what you mean by python in chromium however, I assume, pyiodide may already be doing this.

[mirekrusin]

They did wasm to not do it and have it at the same time.

What they can do is to provide GC to unbloat a lot of those runtimes I guess.

[a-dub]

i don't know the story under wasm, but i looked into what it would take to embed python into a browser years ago.

the hard part at the time was obviously all the hooks between the dom and the javascript runtime as well as concurrency story. python 2 was not built to be driven by callbacks, which is how the whole browser/javascript ecosystem works.

[questiondev]

web assembly is making leaps. looks great

[pjmlp]

Nah, just catching up with what was already possible with ActiveX, Applets, Flash and PNaCL, just more cross browser and political acceptance across all parties.

[pyuser583]

That’s nice.