PyPy.js: Python in the web browser

[amasad]

It's really awesome to see how far this has come. I believe we at Repl.it were the first to try something like this in production. We emscriptined CPython to JavaScript and contributed quite a bit to the project in the process (including the initial virtual filesystem implementation).

https://github.com/replit-archive/empythoned

I deployed this at scale at Codecademy were millions of users were using it to learn Python but hit a lot of problems with it. For one, if you're in a 3rd world country then the bundle size is a non-starter (this is potentially solved with WASM being a binary format). Even if you managed to download the bundle lots of old computers would run out of memory trying to parse the JS (again probably solved by the binary AST format of WASM). Because of this and a few other issues (the dev experience of emscripten was really hard) we had to move away to running code on remote containers.

Now that I'm back working on Repl.it full time I'm excited to play again with the tech and see what we can do with it this time around.

[flanbiscuit]

I just wanted to say how much I love Repl.it and use it all the time. I use it as a testing scratchpad mostly but I know it's much powerful than that. I appreciate its simplicity. Thanks for a great product!

[jumasheff]

Thank you for your work at Codecademy -- a place where I learned how to program. I'm working through my 4th year as a developer :)

[hathawsh]

I'm curious about the virtual filesystem:

  Welcome to PyPy.js!
  >>> import os
  >>> os.listdir('/')
  ['tmp', 'home', 'dev', 'lib']
  >>> f = open('/what', 'w')
  >>> f.write('hey')
  >>> f.close()
  >>> os.listdir('/')
  ['tmp', 'home', 'dev', 'lib', 'what']
  >>> open('/what').read()
  'hey'

What is in the stack that makes that work?

[mattigames]

The Emscripten File System API, "The API is inspired by the Linux/POSIX File System API, with each presenting a very similar interface." https://kripken.github.io/emscripten-site/docs/api_reference...

[kjeetgill]

I believe emscripten provides a virtual filesystem interface. Since it's essentially translating/running c it's either hooking through glibc or the syscall interfaces.

[matmann2001]

    >>> import os
    >>> for root, dirs, files in os.walk('/'):    
    ...   for d in dirs:
    ...     print os.path.join(root, d) + '/'
    ...   for f in files:
    ...     print os.path.join(root, f)
    ... 
    /tmp/
    /home/
    /dev/
    /lib/
    /home/web_user/
    /dev/shm/
    /dev/null
    /dev/tty
    /dev/tty1
    /dev/random
    /dev/urandom
    /dev/stdin
    /dev/stdout
    /dev/stderr
    /dev/shm/tmp/
    /lib/pypyjs/
    /lib/pypyjs/lib_pypy/
    /lib/pypyjs/lib-python/
    /lib/pypyjs/lib_pypy/encodings/
    /lib/pypyjs/lib_pypy/UserDict.py
    /lib/pypyjs/lib_pypy/__future__.py
    /lib/pypyjs/lib_pypy/_abcoll.py
    /lib/pypyjs/lib_pypy/_structseq.py
    /lib/pypyjs/lib_pypy/_weakrefset.py
    /lib/pypyjs/lib_pypy/abc.py
    /lib/pypyjs/lib_pypy/base64.py
    /lib/pypyjs/lib_pypy/code.py
    /lib/pypyjs/lib_pypy/codecs.py
    /lib/pypyjs/lib_pypy/codeop.py
    /lib/pypyjs/lib_pypy/copy.py
    /lib/pypyjs/lib_pypy/copy_reg.py
    /lib/pypyjs/lib_pypy/genericpath.py
    /lib/pypyjs/lib_pypy/getopt.py
    /lib/pypyjs/lib_pypy/linecache.py
    /lib/pypyjs/lib_pypy/os.py
    /lib/pypyjs/lib_pypy/posixpath.py
    /lib/pypyjs/lib_pypy/pwd.py
    /lib/pypyjs/lib_pypy/re.py
    /lib/pypyjs/lib_pypy/repr.py
    /lib/pypyjs/lib_pypy/sre_compile.py
    /lib/pypyjs/lib_pypy/sre_constants.py
    /lib/pypyjs/lib_pypy/sre_parse.py
    /lib/pypyjs/lib_pypy/stat.py
    /lib/pypyjs/lib_pypy/struct.py
    /lib/pypyjs/lib_pypy/traceback.py
    /lib/pypyjs/lib_pypy/types.py
    /lib/pypyjs/lib_pypy/warnings.py
    /lib/pypyjs/lib_pypy/weakref.py
    /lib/pypyjs/lib_pypy/encodings/__init__.py
    /lib/pypyjs/lib_pypy/encodings/aliases.py
    /lib/pypyjs/lib_pypy/encodings/ascii.py
    /lib/pypyjs/lib_pypy/encodings/base64_codec.py
    /lib/pypyjs/lib_pypy/encodings/hex_codec.py
    /lib/pypyjs/lib_pypy/encodings/latin_1.py
    /lib/pypyjs/lib_pypy/encodings/raw_unicode_escape.py
    /lib/pypyjs/lib_pypy/encodings/string_escape.py
    /lib/pypyjs/lib_pypy/encodings/unicode_escape.py
    /lib/pypyjs/lib_pypy/encodings/unicode_internal.py
    /lib/pypyjs/lib_pypy/encodings/utf_16.py
    /lib/pypyjs/lib_pypy/encodings/utf_8.py
    /lib/pypyjs/lib-python/2.7/

[lucb1e]

The speed is surprisingly reasonable. Testing with a simple loop, I get about 1.3-1.4 million loops per second in python3, 2 million loops per second in python2, 14.5 million loops in pypy, and 0.61 million loops in the browser.

My code basically calls int(time.time()) until its value changes, doing i+=1 for every time it did not change (n=10 for every platform (python2/3/pypy/browser)). My browser is Firefox 61, full code here: https://hastebin.com/turenofise.py

[shuzchen]

Fetching time in the browser isn't accurate because a lot of trackers exploit timing in order to fingerprint users. A lot of browsers now purposefully reduce the precision of the clock. See:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

https://johnresig.com/blog/accuracy-of-javascript-time/

[alexchamberlain]

I wonder if the time.time is the slow part there? Maybe worth doing a busy look for say 5 million iterations, timing that then doing the calculation for loops per second.

[tim333]

Though trying to add the numbers for 1 to a million in a loop javascript was about 10 - 20 times faster when I tried it.

[philipov]

Which version of python 3 did you test with? I expected 3.6+ to be faster than python 2.7

[jwilk]

This pastebin requires JS...

Here's the code:

  import time
  a=-1
  j = 0
  i=0
  while j < 10:
      if a!=int(time.time()):
          print(i)
          a = int(time.time())
          i=0
          j += 1
      i += 1

[asmosoinio]

There's some pypy.js vs cpython benchmarks at http://arewepythonyet.com/ - those look very good as well.

[veganjay]

There is a similar Python interpreter here: http://www.codeskulptor.org/

It is used by a class taught on Coursera (Introduction to Interactive Programming in Python)

From what I can see:

- PyPy.js benefit: runs faster

- codeskulptor benefit: can use GUI components

[adamwiggins]

Great project. My team used it for a live-coding Python notebook: https://github.com/inkandswitch/livebook#readme

To my surprise it was faster than Jupyter / CPython for some tasks (e.g. loading and parsing large CSVs).

[dunpeal]

Shouldn't be too surprising, given how PyPy is generally faster than CPython by about x7-8 factor:

http://speed.pypy.org/

I'd expect a much larger difference in tasks that are pure Python (loading a CSV is done primarily through the csv module, i.e. all the hot loops are in C).

[benatkin]

Wow, that's awesome! Direct link to the live notebook: https://livebook.inkandswitch.com/fork/welcome

Naturally, I changed the first code block to this:

    from random import randint, seed
    seed()
    "\n".join([randint(5, 25) * ' ' + s for s in ['such code', 'much live', 'wow']])

[bmays]

This is great, was looking for this exactly!

[VectorLock]

I'd love there to be a client-side webapp development system using Python with feature and output parity to JavaScript. If it was a product, I would buy it.

[tony-allan]

Pity it's not Python3.

[rtkwe]

Yeah weird. I would expect Python2 to be dying off more by now. Wonder if we're headed to an environment where there's essentially 2 separate languages as Python 3 continues to change and grow?

[jchw]

Assuming the work here was primarily the asm.js based emitter, maybe it can be ported to PyPy3 without needing to go from scratch.

[wietze]

The most important import doesn't work, unfortunately:

  import antigravity

[cybersol]

At least all is not lost:

  Welcome to PyPy.js!
  >>> from __future__ import braces
    File "<console>", line 1
  SyntaxError: not a chance
  >>>

[cecilpl2]

That was the first thing I tried :)

[duhi88]

This is pretty old, and hasn't seen an update in over a year. I never understand why links links this get posted with no discussion to start it off with.

Also, wouldn't this be better implemented with WASM?

[stev0lution]

Maybe something like this? https://github.com/iodide-project/pyodide

[whalesalad]

No good deed goes unpunished.

[toyg]

https://www.xkcd.com/1053/

[anarchy8]

> This is pretty old, and hasn't seen an update in over a year

So pretty much like every Python project then. What's happening to the Python community?

[costrouc]

Will web assembly have any effect on a future web based python interpreter? Is there one in the works?

[aylmao]

Someone posted this on another thread: https://github.com/iodide-project/pyodide

What I gather is that it's compiling CPython to WASM and ships it along with some other utilities. It looks pretty cool.

[ipsun4]

Tried Fibonacci, and the thing crashed for 100. I'm not sure that a wasm python library can do any heavy lifting.

[jokoon]

seems brython is more appropriate, mature and usable.

[klhugo]

I tried to do a little something with Emscripten and WebAssembly, but I stopped on the sockets implementation. It relies on WebSockets. I wonder how well we could emulate real sockets using WebRTC to make WebAssembly apps more sophisticated.

[askvictor]

Is there an up-to-date/maintained comparison of python-in-the-browser systems? As well as this, there's brython and skulpt, and it would be nice to be able to compare various implemented features, quirks and speed.

[imh]

Really amazing. I use python every day at work, and most of the crucial libraries have tight inner loops written in C or Cython. I wonder what the web assembly future holds for those languages, and inter-language FFI.

[slezyr]

    >>> os.getlogin()
    'root'

Nice joke :)

[cutler]

  File "<console>", line 2
    return f'{fname} {surname} is {age}.'
           ^

SyntaxError: invalid syntax

So this is a subset of Python or something like that?

[Nicksil]

I believe this feature was introduced in Python 3.6. This implementation may be 3.6 <

Edit: Just checked the site and see it uses PyPy which states -- on their site -- the python version is 3.5.3

[nichochar]

This is awesome. They mention that it is a "compliant" implementation of python. How does one go about proving this?

[tathougies]

There's no python standard, but my guess is that-- since this is based on the same source code that native PyPy is, with a different compiler backend -- they are making the claim based on the fact that native PyPy is widely considered to be similar enough to CPython for many workloads. Thus -- assuming the correctness of the RPython -> JavaScript transformation (which they have not proven, and would be a monumental task) -- you should expect this version to work like the native PyPy.

[parabx]

this is pretty neat! I was wondering, are there any projects to transpile python to js, but targeting the server side? It would be interesting to port a flask application, for example, to run on node and benefit from the jit, or maybe use express and call flask through its wsgi interface.

[icebraining]

You're better off running it on the original version of PyPy, which already uses a JIT.

[Skunkleton]

Why not just run these in their native environment?

[nautilus12]

Why use this instead of jupyter? Some educational applications maybe?

[antoineMoPa]

I think jupyter works with a backend server to execute python (right?).

[ealhad]

This runs directly in the browser, whereas Jupyter needs a server.