Making Python fast – Adventures with mypyc

[ok_dad]

I’ll be that guy who says I love Python but it’s been shoved into too many spaces now. It’s been a great tool for me for writing things that require a lot of I/O and aren’t CPU bound.

I am even rethinking that now because I was able to write a program in Go with an HTTP API and using JSON as the usual API interchange format in one night (all stdlib too), and it was so easy that I plan to pitch using it for several services we need to rewrite at work that are currently in Python. That would be very similar to what I wrote in a day.

If Python doesn’t fix their packaging, performance, and the massive expansion in the language, I think it’s going to start losing ground to other languages.

[intelVISA]

Can't fault Py as a scripting lang but the lengths people go to kludge it into areas it shouldn't really be (in prod at least) is always a massive red flag for any org.

[eru]

Microsoft is bankrolling some efforts to improve Python performance. They even hired Guido for that.

(Disclosure: I'm a very minor volunteer contributor to that effort. I have a series of pull requests in the works that will improve runtime performance by about 1% across the board. I also have some more specialised improvements to memory.)

[jeremycarter]

Syntactically Python is great. Runtime though, its not. Its fast to code in - that is all.

[rkrzr]

I didn't know that you can compile individual modules with mypyc. That's very interesting since it allows a gradual adoption of the compiler, which really helps with big codebases.

Do you know if there are any requirements for which modules can be compiled? E.g. can they be imported in other modules or do they have to be a leaf in the import tree/graph ?

[traverseda]

Having read through the docs Mypyc has a concept of "native classes" and python classes, and it looks like you can use a "native" (compiles) class from regular python and vice-versa.

So my reading is that it should be pretty seamless.

[meadsteve]

I recently experimented with using mypyc to make some of my python a little faster. I was pleasantly surprised with how well it worked for very little code change so I thought I'd share my experiences.

The blog post wanders around a little because I had to add setuptools and wheel building as my project had previously skipped this.

[4140tm]

I just found out about Lagom from this blog post and it's exactly what I have been looking for.

All other Python options I've seen feel too involved or leak too much into your code. Lagom seems to balance everything just right.

Thank you!

[atoav]

Anybody using Python and Rust should also check out maturin and pyo3. I run some (non public) Python modules created in Rust and both the performance and the testability is stellar.

[meadsteve]

Yeah these are great approaches too. I'd actually considered a rewrite of the core in rust before I went with mypyc. But it was nice not to have to do a rewrite.

[jblindsay]

I have the exact same experience. Both Maturin and PyO3 have been a game changer for the work that I have been doing lately. It works so seamlessly.

[kodablah]

We built the logic backing the Temporal Python SDK[0] in Rust and leverage PyO3 (and PyO3 Asyncio). Unfortunately Maturin didn't let us do some of the advanced things we needed to do for wheel creation (at the time, unsure now), so we use setuptools-rust with Poetry.

0 - https://github.com/temporalio/sdk-python

[bsenftner]

Worth mentioning Taichi, a high-performance parallel programming language embedded in Python. I've experimented with it a bit, and high-performance is very true. One can pretty much just write ordinary Python, plus enhancing existing Python is not that difficult either.

From their docs:

You can write computationally intensive tasks in Python while obeying a few extra rules imposed by Taichi to take advantage of the latter's high performance. Use decorators @ti.func and @ti.kernel as signals for Taichi to take over the implementation of the tasks, and Taichi's just-in-time (JIT) compiler would compile the decorated functions to machine code. All subsequent calls to them are executed on multi-CPU cores or GPUs. In a typical compute-intensive scenario (such as a numerical simulation), Taichi can lead to a 50x~100x speed up over native Python code.

Taichi's built-in ahead-of-time (AOT) system also allows you to export your code as binary/shader files, which can then be invoked in C/C++ and run without the Python environment.

https://www.taichi-lang.org/

[intelVISA]

> export your code as binary/shader files

That's cool af.

[raphaelrk]

Speaking of python performance, I recently benchmarked "numpy vs js" matrix multiplication performance, and was surprised to find js significantly outperforming numpy. For multiplying two 512x512 matrices:

    python
      numpy:               ~3.30ms
      numpy with numba:    ~2.90ms

    node
      tfjs:                ~1.00ms
      gpu.js:              ~4.00ms
      ndarray:           ~118.00ms
      vanilla loop:      ~138.00ms
      mathjs:           ~1876.00ms

    browser
      tfjs webgpu:          ~.16ms
      tfjs webgl:           ~.76ms
      tfjs wasm:           ~2.51ms
      gpu.js:              ~6.00ms
      tfjs cpu:          ~244.65ms
      mathjs:           ~3469.00ms

    c
      accelerate.h:         ~.06ms

Source here: https://github.com/raphaelrk/matrix-mul-test

[netheril96]

Numpy runs on CPU. Tfjs runs on GPU. Not a fair comparison.

[sampo]

TensorFlow.js matrices are immutable, which puts more restrictions on your programming style that standard Numpy. You cat get immutable, GPU-enhanced matrices for Python, too.

[wcdolphin]

Is anyone else using MyPyC in production and can share their experience? Did you attempt the compile it all approach, or incrementally add? What do compile times look like at scale?

Would love to buy you a coffee and hear about your experience and the challenges and benefits.

[intrepidhero]

I like the concept of using mypyc to leverage type hints to compile python. But I was pretty frustrated recently when I got bit by a bug in mypyc[1] while trying to use black. Especially since I wasn't using mypyc myself and so didn't realize it was even in my dependency tree. Beware adding "alpha" quality software as a dependency to your supposedly production ready tool.

[1] https://github.com/psf/black/issues/2846

[mrtranscendence]

Nice to see this. Do they have a project roadmap for mypyc?

Doubling performance is nice, though it does still leave a lot of performance on the table. I’d be curious to see a comparison between this and Cython.

[chrisseaton]

> for free ... this was a problem as a number of my tests rely on this [incompatible behaviour]

[meadsteve]

Maybe I should have said for cheap

[an1sotropy]

I'm curious how to compare this with a PyPy FAQ: https://doc.pypy.org/en/latest/faq.html#would-type-annotatio... which describes a bit about why type hints aren't as helpful to optimize code under PyPy as one (including myself) might think.

Can someone explain more about how mypyc is in a better position to produce better optimizations than pypy, or am I confused about this?

[detaro]

pypy argues that considering type annotations gives them less useful data than their existing tracing does, and thus pypy wouldn't be faster if it considered them. Something like mypyc by design has no chance of doing tracing, and thus has to work with annotations. (I also don't see where you get the claim from that that mypyc has better optimizations than pypy? But the two also follow different designs, so they might be good at different things)

[BiteCode_dev]

Mind you, it still requires to have a c compiler, to be installed separately. It's very easy on linux, but a x-code install on mac, and can be fiddling on windows.

Still nice, but not like golang or rust where you have a stand alone solution.

It's an alternative to nuitka, which I recommend to try out.

[bobsmooth]

What's with this fascination with making python fast? It's not supposed to be fast, it's supposed to be simple. If you want speed use a compiled language. Trying to make python fast is like trying to strap a turbocharger to a tricycle.

[alanwreath]

I agree with you -- but I also don't say no to free food.

I mean regardless of whether mypy was going to make my code run faster I would have used it for the shear confidence it gives wrt to my code correctness. The fact that I can use that same code (untouched) to speed it up... that's just means I get to have my cake and eat it too :P

[pizza]

Python programming is simple because I the programmer have to do less stuff. But also, to a really really crude approximation, some asm that does less stuff is both simpler, and faster, than some asm that does more stuff (depends on what it is I'm doing in particular, but, it's a decent rule of thumb).

But there is a disconnect between Python programming simplicity and Python speed, which stems from the fact that under the hood Python is doing much more than its minimal asm 'spiritual equivalent'.

But in a pure abstract theory sense, it shouldn't "need" to. I don't really care about the intricacies of garbage collection or global interpreter lock or page misses etc - what if I just care about "can I make this nice idea into reality in 10 minutes". The reality is that I'm just barely working with the tip of an iceberg composed of 60+ years of computer abstractions. But who can blame me - I am but a mere mortal.

If we could have a programming language that is both simple for the programmer and simple for the computer, it would be great. It's not that unreasonable that people start from the user experience side - making a simple language faster, by getting rid of unnecessary work - rather than the opposite extreme: making it simpler to come up with optimal machine code whose simplicity withstand contact with hundreds of vestigial appendages that just have to be dealt w bc of computer history (spanning from how to do a syscall to how to make a gui in some particular os)

[nequo]

I can think of use cases in academic research for example.

Many pieces of code run only a handful times but potentially move a lot of data. Sometimes reimplementing existing code in C/Haskell/Rust, including finding equivalent libraries, writing tests, and debugging, only because the computation turned out to be heavier than I had expected is not a good use of time. If that’s the place where I’m at, PyPy, mypyc, etc., might just do the trick.

[peterkelly]

mypyc is cool and all, but I can't help thinking about how Node just JITs everything automatically without the need for any special steps like this.

[chrisseaton]

That's not Node - that's V8. And it's possible to do the same thing for Python - there's nothing magic about JavaScript compared to Python - it's just a lot of engineering work to do it, which is beyond what this project's scope is. PyPy does it, but not inside standard Python.

[BiteCode_dev]

That's what Microsoft is paying Guido for, for the next versions of python.

[staticassertion]

I'm curious - if you can't have nested classes, does @dataclass(slots=True) not work?

[pmarreck]

Have they cleaned up Python's packaging/dependency problem yet?

[talideon]

You're going to have to define what you mean by that first, given there Python packaging landscape has changed quite a bit in the past few years.

[kingkongjaffa]

Can this work with pyinstaller to make an executable faster?

[Cyphase]

I can't see why not. I've packaged some complex dependencies with PyInstaller – on Windows. There is always a way. This wouldn't even be particularly difficult.