Python 3.11 in the Web Browser

[modeless]

I really, really wish it were possible for wasm modules like this to be shared between sites. That is, python.org could host an official wasm module, and the first time you visited a site using python the browser would download and compile it once and cache the result, and then any other website could use python with no download size or load time penalty. But unfortunately privacy concerns killed this utopia. Caches are now partitioned per origin, so every site that uses python has to download and compile the wasm module separately even if it's the exact same module from the same place, and that is unlikely to ever change: https://developer.chrome.com/blog/http-cache-partitioning/

[nine_k]

While I understand how finding a particular unique resource cached may allow to glean some privacy-infringing info, I don't see how detecting a common library downloaded from a site that provides it for everyone (like python.org) would help reveal anything.

If users might define a list of sites trusted for cross-site caching (like fonts.google.com and other known library CDNs), this could help cache quite some common resources without downsides to user's privacy.

[xpressvideoz]

Wait, so the traditional "use the JavaScript CDN as a cross-site cache" method does not work anymore? How did I not know that!

[skybrian]

I wonder if a browser vendor could host a repository of common libraries? With a limited number of versions available that changes only at the same time as the browser, maybe it wouldn't leak much info. It would be almost like including the code in the browser.

[cgb223]

Couldn’t browsers theoretically “bake in” the python.org binary to the browser itself to avoid this?

Kind of like how macOS (used to) ship with python preinstalled?

[fnord123]

While I get you point, I think it's of limited usefulness for Python. You would end up with Snekr.io using 3.11 and cheeseshoppr.ws using Python 3.8.4 and etc. You can get some packages for any Python3 but they are the smaller ones. The larger wheels like numpy and pandas have C code which needs to be compiled or there's a giant version dependent wheel.

Server side Python is often deployed in a venv which has the desired python version and all the dependencies copied with the code.

[krapp]

That page says there's a model for shared libraries being considered, so maybe there's hope.

[froh]

You'd need trusted providers, and a mechanism to 'import' a trusted something into your 'domain'.

What am I missing?

[bitpow]

You’re basically describing the old Java plugin paradigm.

[mcintyre1994]

I’m a big fan of Python in WASM! It really reduces the friction of playing around with things. Something I find pretty useful that I hacked together using it is https://pyground.vercel.app/ - I use it whenever I have a CSV or JSON file I want to do some poking around in using Python.

You can just drag and drop a file in, it’ll be imported in as the ‘data’ variable, and then you can run Python code/do matplotlib visualisations without installing anything.

[discordance]

Jupyter in WASM is a thing too:

https://blog.jupyter.org/jupyterlite-jupyter-%EF%B8%8F-webas...

[wishawa]

Have you seen Skulpt [1]? It doesn’t do matplotlib (or most other library) but it’s pure JS.

[1]: https://skulpt.org/

[productceo]

Interesting! So I can write any Python code here in the web app, and the Python code will run locally on my client machine without sending any information to 3P web servers?

[irisman]

Did you use tailwind css for that?

[rewq4321]

Those interested in this should check out Pyodide[0]. It basically "just works" so long as the libraries you import are pure Python or are part of the core scientific stack (the Pyodide authors have done the work to manually port all the C code behind numpy, scipy, etc.).

What I really wish for is for ~all Python packages to work in the browser without manual porting of the underlying C/Rust/etc. being needed, since a lot of the interesting and useful libraries aren't pure Python, and manual porting is non-trivial.

I'm not sure what the best route to that future is, but I'm guessing it'd probably help if Python had a wasm runtime in its standard library[1], since then authors of libraries that use C/Rust/etc. might make cross-platform builds (perhaps by default).

Regarding this Pycon speech, it seems that it's related to the following entry in the 3.11 changelog[2], which the speaker was heavily involved with:

> CPython now has experimental support for cross compiling to WebAssembly platform wasm32-emscripten. The effort is inspired by previous work like Pyodide. (Contributed by Christian Heimes and Ethan Smith in bpo-40280[3])

But maybe Christian has more to reveal here? In any case, I'm hugely appreciative of all the work that is being done to bring Python to the browser!

[0] https://github.com/pyodide/pyodide

[1] https://discuss.python.org/t/add-a-webassembly-wasm-runtime/...

[2] https://docs.python.org/3.11/whatsnew/3.11.html

[3] https://bugs.python.org/issue40280

[kaba0]

I think the closest to that goal is GraalVM. It can run LLVM bit code for the C parts while can natively run Python. Since the whole thing is java and java byte code can be compiled to wasm/js by teavm it should work even know though it is definitely not streamlined yet.

[heavyset_go]

I think it would be cool if browsers shipped with WASM-compiled interpreters and runtimes for other languages, like they do with JavaScript.

That way you'd be able to use other languages in the browser without needing users to download 20MB of a WASM-compiled interpreter just to run 1KB of code.

[klodolph]

You're still downloading the 20MB WASM-compiled interpreter, it's just that now you're redownloading it every time your browser updates.

[tester756]

>WASM-compiled interpreters and runtimes

in which version?

[999900000999]

I'd love that too, if I was a better programer I'd likely fork Chromium and build in Python support.

[doctoboggan]

I did a quick speed comparison:

   import numpy as np
   import time
   n = time.time()*1000
   r = np.random.rand(10000000)**2
   print('done in', time.time()*1000-n, 'ms')

In the browser I was getting around 200ms and on my computer I was getting around 65ms, so around 3 times slower.

Also, if I try to allocate an array of length 100,000,000 then I get a `MemoryError` in the browser.

Does anyone know any ways to get around these limitations?

[stavros]

I find WASM in general very cool. Is there a way to run a WASM "binary" locally? For example, can I distribute a program as WASM and execute ./mywasmprogram and have it run?

I imagine I need a WASM runtime installed, and to somehow get my shell to recognize that WASM programs should be loaded with it (for lack of a hashbang line), but is that actually doable?

[simonw]

Yes, definitely. Wasmer is one way to do that: https://docs.wasmer.io/ecosystem/wasmer/getting-started

[gmatejka]

You can also use node to execute webassembly programs.

You do have to create a js host file, load in your webassembly and then run it with node.

[Klasiaster]

wasmtime, Standalone JIT-style runtime for WebAssembly, using Cranelift: https://github.com/bytecodealliance/wasmtime

wasm3, WebAssembly interpreter: https://github.com/wasm3/wasm3

… and many more

[pjmlp]

The irony is that in the old plug-in days ActiveState did had a Python plugin for the browser.

[jeremycarter]

We've gone full circle

[Too]

If you are ready for some inception, go watch David Baezley use Python to implement a complete WASM interpreter in under 1h. https://www.youtube.com/watch?v=VUT386_GKI8 One of the best coding presentation i've seen.

This should make it possible to run python in wasm in python.

[nabakin]

Found his work https://github.com/tiran/cpython/tree/wasi/Tools/wasm

[teddyh]

> there is one place that Python has not yet conquered: the browser

Well, there was that one web browser written in Python, with built-in Python scripting support: Grail.

[asiachick]

I don't know what python's async support is like but the biggest issue with porting to the web is the browser mostly requires that you exit events before it will do anything. This is contrast to pretty much all other environments. That means there are tons of patterns and libraries that just don't work and require major re-writes or big AST manipulations like asyncify from emscripten.

I'm not saying you can't do async work in any language. Rather I'm saying that likely the patterns you're used to using in some language that's not JavaScript won't work in the browser and you'll have to massively change your style.

[kumarvvr]

I can't wait for the browser to be released from the shackles of JS.

Just imagine how awesome would it be to use the full power of Python to build a react like library.

It is a source of great disappointment to me that WASM cannot directly control the DOM.

[rewq4321]

Why does it need to be direct? A bit of glue code doesn't hurt, especially since most of the other glue is going away with WASI.

[laerus]

WASM Interface types will fix that

[rubenfiszel]

This is huge and comes at the right moment for me. The Web Browser part is far less important to me than being able to compile programs into small and autonomous wasm binaries. I am indeed building an open-source (soon) business app platform based on code: https://windmill.dev

Right now you can define any scripts and run them, but behind the scene, the way the workers work is that they have a static list of dependencies they can handle and always fork a python process to run the code in that environement.

I was scratching my head about how to provide proper isolation and handling of custom dependencies for Python short of zipping the entire list of pip dependencies and unzipping it at runtime. For Typescript, this can be achieved easily using deno compile. Store the small output bundle and run it with the proper runtime restrictions. With the ability to do more or less the same Python, this is a huge game changer.

[jokoon]

How large is the wasm binary?

I would guess it's only for core python, without modules.

Brython was nice to use, although its dom syntax was a bit awkward.

[ksr]

Can someone explain to me, really slowly please, how to take an existing Python codebase such as https://github.com/infojunkie/mma and run it in the browser using one of the technologies mentioned in this thread? Especially, how to deal with filesystem calls that expect some files to be present in various folders. Thanks!

[Skyy93]

According to the documentation and discussion about this feature some unit-tests are currently simple skipped: virtual filestem by Emscripten that's not fully POSIX compliant, no processes, no sockets, async only via the browser event loop (source: https://bugs.python.org/issue40280)

This would mean to me they simply do not work or only to an extend. As you can reading the virtual filesystem seems not fully compatible.

[syrusakbary]

I’m super excited about this being integrated upstream.

Long ago we compiled Python 3.6 and published to WAPM: https://wapm.io/python/python (you can run it online there!)

I wonder if we could publish the new version also! I think we got the Python repl properly running so it would be interesting trying to have that working on the new version too

[jonititan]

Does the talk do a comparison to Pyodide? Seems very similar and I'd be very interested to hear what the advantages and disadvantages are.

[fortzi]

I worry that we’ll now see more broken web apps, given the lack of type safety in Python. I know that JS doesn’t guarantee that either, but at least you can choose to work with TypeScript. True, there’s mypy and alike, but their hard to integrate into existing projects, and are also not perfect.

[madacol]

type hints[0], mypy[1] provide type annotations to python

And type hints is something that JS lacks. How many times I've tried to copy paste typescript code to nodejs/browser console and fail miserably

[0] https://docs.python.org/3/library/typing.html

[1] http://mypy-lang.org/

[z3t4]

type annotations does not guarantee correctness, performance, nor does it make your code bug free. Type annotations however helps when writing code, as the tooling will know that something annotated as type X have these prototype variables, or that the object annotated with interface should (but does not guarantee) to have the annotated interface members. This can also be achieved via type inference, but inference is more complicated then simply annotating (writing down) the type in the code. Before Typescript some people used to have the type in the variable name, like strName, fltCost, arrPeople. Bugs are detected by running the program in a chaotic environment (live/prod) where there are humans involved. Bugs can also be found by carefully and critically studying the code. Some bugs can also be found by inference or type annotation tooling, or usually when trying to run the code for the first time.

[KolenCh]

Sorry, don’t quite follow, why would the lack of type safety in Python make this a problem, and how would Typescript help in this situation?

This comes from Python dev with no idea how to make web app. Eg I’ve seen some package not maintained for years in PyPI while continue to work fine.

Also I think Python is strongly typed while js isn’t. I would think that’s a bigger problem?

[weatherlite]

People who want Python on the browser would have probably not chosen Typescript anyway, it's mostly not the same demographic. Not everyone cares about types.

[tomatowurst]

when can we expect a recording of this to be uploaded on youtube, are there any existing content out there that talks more about wasm and python? seems really important, especially for many researchers and data scientists who use python primarily to be able to cross over to the browser.

[da39a3ee]

Is there any way to compile a subset of Python itself to Wasm? (As opposed to compiling its interpreter to Wasm.) Same question for Javascript.

[tyingq]

WASM is a virtual machine, but it's missing support for some things that would make that practical. GC support, polymorphic inline cache, char/byte types, reference types, etc. Some of that is on the WASM roadmap.