PyPy.js: A fast, compliant Python implementation for the web

[btown]

This is glorious. From https://github.com/rfk/pypyjs/blob/master/CONTRIBUTING.rst :

> We have the following major components in the PyPy repo:

>> An "emscripten" build platform definition, which teaches pypy's rpython toolchain how to compile things with emscripten: ./deps/pypy/rpython/translator/platform/emscripten_platform/.

>> An rpython JIT backend that emits asmjs at runtime: ./deps/pypy/rpython/jit/backend/asmjs/.

>> A "js" builtin module for the resulting interperter, to allow interaction with the host javascript environment: ./deps/pypy/pypy/module/js/.

>Along with these wrappers to help working with the resulting interpreter:

>> A wrapper to load up the compiled VM and expose it via a nice javascript API: ./lib/pypy.js.

>> A script for bundling python modules into an indexed format that can be easily loaded into the browser: ./tools/module_bundler.py.

In https://github.com/rfk/pypy/blob/master/rpython/jit/backend/... alone, there are 2887 lines of custom Python code to emit asm.js for function and block JITing.

And at 3.26 seconds to initialize from cache on a 4-year-old machine, 5MB gzipped (not too much worse than your average high-quality gif) and only 1.5x slower than CPython... this is actually approaching viable if you needed to run Python business logic in the browser. Which I never thought I would see.

[brachi]

I watched the presentation in Pycon 2015[1], and I really liked how honest/aware they are about the challenges and tradeoffs (so far, the performance it's pretty bad compared to javascript, and the download size is still an issue). At the end is answered why python can't be directly included in Firefox or other browsers(the presenter/main developer works for Mozilla)

[1]https://www.youtube.com/watch?v=PiBfOFqDIAI

[itsnotlupus]

For historical context, early IE browsers weren't hardcoded to use JavaScript. We know about VBScript, but really there was a generic framework called Active Scripting that would allow any other language to be plugged in.

Python was one of those languages. You could download an ActivePython distribution and start being able to run python in your web page with a <script language="PythonScript"> tag.

I'm not sure if this still works in current IE versions. It might.

[nailer]

> At the end is answered why python can't be directly included in Firefox or other browsers(the presenter/main developer works for Mozilla)

Direct link to answer: https://youtu.be/PiBfOFqDIAI?t=1611

[voltagex_]

I wonder if Python could be embedded as an addon.

[vog]

> python can't be directly included in Firefox or other browsers

So Python is a bad candidate, but there are better candidates out there. For example, jsofocaml (using OCaml in the browser) looks quite promising.

In the end, we need a lightweight yet properly defined and highlevel language, and the ML languages as well as the LISP languages are great candidates for that.

[bayesianhorse]

Besides the big download size, it does several things very "right": Full Python data model, access to the full javascript model (albeit slowly currently).

Also PyPy is the right choice as the basis for this (as opposed to CPython) because its ecosystem is generally more "pure" Python than CPython's. C Extensions would only get in the way.

[fijal]

it's systematically translating the PyPy interpreter to the Javascript, so it has to get semantics right. However I'm not sure if the drawbacks can be addressed in the current JS model, notably: very slow warmup, huge download size (that's primarily PyPy fault though), the lack of good JS access.

[ggchappell]

> ... access to the full javascript model ....

How is the JS model accessed?

[jedberg]

So does this mean that since I'm better at Python than Javascript, I can now use this to make my web frontends in Python? Because that would be pretty sweet.

[strange_quark]

That would be great as one of the things that puts me off of front end development is lack of language choice. I adore Python as a language and it would certainly make a great choice for a front end language if supported. Maybe if this project takes off, browser vendors would ship a Python interpreter alongside the JS one. I'd bet that Apple, Google, Mozilla, etc. could make Python fast if they threw as much money at it as they do their JS interpreters. And PyPy is already pretty fast -- it's certainly way faster than JS engines were before V8 and JavaScriptCore hit the market.

This will probably never happen, but I can dream.

[riffraff]

weren't you already able to do it with pyjamas/pyjs?

[est]

Yo know the most troubling aspect of browse side scripting is the DOM, js syntax has its issues but not the biggest issue.

[pyvpx]

I had the same thought. "this is cool. but what should I do with it?"

[nosir33]

Sample code to execute some Javascript code inside PyPy.js. I couldn't find any examples.

    import js
    js.eval("console.log(\"hi!\")")

[fragmede]

It's a bit messy, but:

    js.eval('var div = document.createElement("div"); div.style.width = "100px"; div.style.height = "100px"; div.style.background = "red"; div.style.color = "white"; div.innerHTML = "Hello"; document.body.appendChild(div);');

manages to append a div.

--

Directly modifying document.body.innerHTML in any way seems to hang it. Both:

    js.eval('document.body.innerHTML += "<div></div>";');
    js.globals.document.body.innerHTML += "<div></div>"

Cause it to hang, though the javascript console shows it was added (or using an img tag instead).

(ChromeOS v43 beta)

[Retr0spectrum]

Just playing around:

    import js
    c = js.globals.document.createElement("canvas")
    js.globals.document.body.appendChild(c)
    ctx = c.getContext("2d")
    ctx.fillRect(10, 10, 10, 10)

Edit: Here is a simple animation:

    import js
    import math
    import time
    c = js.globals.document.createElement("canvas")
    js.globals.document.body.appendChild(c)
    ctx = c.getContext("2d")
    
    def render():
      ctx.clearRect(0, 0, c.width, c.height)
      t = time.time()
      ctx.fillRect(20+math.sin(t*10)*10, 20+math.cos(t*10)*10, 10, 10)
    
    js.globals.window.setInterval(render, 20)

Here's a version you can paste in properly: http://pastebin.com/raw.php?i=67GadSUV

[dag11]

Or this:

    import js
    js.globals.console.log('hi!')

[stuaxo]

It would be good if they could support a common api with brython, which has "import web" and some other bits...

I guess these things will take a while to shake out, but I'd imagine a python-in-browser-api PEP at some point.

[Nitramp]

I think there is no point in trying to emulate different language semantics on top of JavaScript.

asm.js is a weird hack that lives in a totally different world than actual browser APIs (e.g. the DOM), and you have to emulate your entire runtime and environment to get anything useful, which means your binary is going to be huge (as in this example, but similar things apply).

Emulating better semantics than JS based on JS usually leaves you in this uncanny valley where you either trade off performance and size for nicer semantics (e.g. killing the null/undefined dichotomy, or 64 bit integer math), or you end up with odd semantics that don't quite fit the language you're compiling from, or a mix of both, which is a usability disaster.

So the only real chance to improve on the state of JS is being a syntactical shim on top of JS and help users with better syntax, static analysis, and compile time tooling. That'd be TypeScript.

[Tossrock]

Looks like there are some issues to iron out...

  >>> from datetime import datetime
  >>> datetime.now()
  Traceback (most recent call last):
    File "<console>", line 1, in <module>
    File "/lib/pypyjs/lib_pypy/datetime.py", line 1548, in now
      return cls.fromtimestamp(t, tz)
    File "/lib/pypyjs/lib_pypy/datetime.py", line 1522, in fromtimestamp
      result = cls(y, m, d, hh, mm, ss, us, tz)
    File "/lib/pypyjs/lib_pypy/datetime.py", line 1459, in __new__
      hour, minute, second, microsecond)
    File "/lib/pypyjs/lib_pypy/datetime.py", line 312, in _check_time_fields
      raise ValueError('microsecond must be in 0..999999', microsecond)
  ValueError: ('microsecond must be in 0..999999', 1716000)

[caryhartline]

That works now.

[nighthawk454]

Sadly, "import antigravity" doesn't work

[e12e]

I was more disappointed (but not surprised!) that "import pip" didn't work.

If it did (and it has big ramifications for how much of "python" (for some definition of "python" that includes most of the standard library) works. Eg sockets, interacting with some sort of "local" "file-system" etc) one could (this is handy if working on windows, and running python via cmd-r python.exe):

  import pip
  # pip.main expects a list of arguments, hence .split() 
  pip.main("install ipython".split())
     # Only needed first
     # time, as it installs ipython under site-packages.
     # Only works if the user has write access, obviously

  import IPython
  IPython.start_ipython()

[reuven]

Yes, but "import this" does!

(However, it took about 10 seconds to execute. Maybe that's the network I'm on, but even so...)

[dulvac]

I can imagine a way to include language extensions in the browser and have them loaded up lazily in a sandbox. And a checkbox that says "Allow websites to run code" or something. Then you could develop an extension for your favourite language and either install them manually or have some popular ones bundled with the browser.

If we're going to run everything in the browser, why use javascript as the low-level language that stuff compiles to?

I know portability and existing standards are a limitation, but those can be influenced by the 3 (?) major browsers if needed.

I think asm.js is an awesome cute little thing, but the fact that it's taken seriously is worrisome to me.

Am I completely crazy here?

[bootload]

If software is eating the world, JavaScript is eating the languages.

[zo1]

Only because the browser vendors are too scared (for whatever reason, legitimate or otherwise) to add extra languages to the mix, like python.

[henryaj]

Please humour my lack of knowledge: is there much difference between something like this and Opal[0]?

[0] - http://opalrb.org

[Veedrac]

PyPy.js is not a transpiler, it is a JITted interpreter. This has correctness advantages since the major work is upfront, and potential throughput advantages, but is results in huge upfront download and warmup costs.

[pc2g4d]

Not working for me:

"Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://10.30.229.27:15871/cgi-bin/blockpage.cgi?ws-session=1.... This can be fixed by moving the resource to the same domain or enabling CORS."

Firefox 37.0.1

[acron0]

Obligatory link to Atwood's Law: http://blog.codinghorror.com/the-principle-of-least-power/

Although I appreciate that this isn't technically "written in JavaScript" but it follows the principle.

[MBlume]

I'm curious -- since the custom emitter is tied to the RPython JIT and not to the Python interpreter, could one easily plug in the existing Ruby or PHP interpreters written in RPython and get similar performance?

[estomagordo]

Unfortunately, this is of little use to me. Why? Well, with my Swedish keyboard layout, some alt-combinations are crucial (mainly square brackets, []). And these keypresses do not seem to work.

[Veedrac]

You can try `vm.eval` from the JS console.

[hurin]

>>> def foo(x,y):

... return x,y

debug: OperationError:

debug: operror-type: SyntaxError

debug: operror-value: ('EOL while scanning string literal', ('c callback', 1, 12, "r = c.push('def foo(x,y):\n", 0))

>>>

[acbart]

I use Skulpt for a lot of my research - I'd love to see a comparison against it in terms of speed, accuracy, etc.

[gdw2]

Can the client-side python access the DOM?

[venti]

yes it can: import js js.globals.document.getDocumentById("...")

[dlitz]

Is there a Python 3 version of this?

[venti]

Yes, in his talk at https://www.youtube.com/watch?v=PiBfOFqDIAI&t=1355 Ryan Kelly already showed a Python 3 version of pypy.js

[techdragon]

I was talking with some contributors at PyCon last month and there was a consensus that Python 3 is actually preferable. There's issues to be resolved that are more important than switching the version to 3 so the work is focused on these for now.

It boils down to the argument that "if there is no existing Python code relying on Python 2 semantics, why even bother making a Python 2 version, why not skip straight to Python 3 and not introduce the switching problem later?" There's a small matter of the Python 3 version not being the prime target of PyPy but they are updating it and progressing it forward so it's not any kind of roadblock just a little less than perfect.