It turns out that if you have language semantics that make optimizations hard, making a fast optimizing compiler is hard. Who woulda thunk?
To be clear, this seems like a cool project and I dont want to be too negative about it, but i just think this was an entirely foreseeable outcome, and the amount of people excited about this JIT project when it was announced shows how poorly a lot of people understand what goes into making a language fast.
I was active in the Python community in the 200x timeframe, and I daresay the common consensus is that language didn't matter and a sufficiently smart compiler/JIT/whatever would eventually make dynamic scripting languages as fast as C, so there was no reason to learn static languages rather than just waiting for this to happen.
It was not universal. But it was very common and at least plausibly a majority view, so this idea wasn't just some tiny minority view either.
I consider this idea falsified now, pending someone actually coming up with a JIT/compiler/whatever that achieves this goal. We've poured millions upon millions of dollars into the task and the scripting languages still are not as fast as C or static languages in general. These millions were not wasted; there were real speedups worth having, even if they are somewhat hard on RAM. But they have clearly plateaued well below "C speed" and there is currently no realistic chance of that happening anytime soon.
Some people still have not noticed that the idea has been falsified and I even occasionally run into someone who thinks Javascript actually is as fast as C in general usage. But it's not and it's not going to be.
Especially when one keeps ignoring the JITs of dynamic languages, that were in the genesis of all high end production JITs being used nowadays, tracing back to Smalltalk, Self, Lisp, Prolog.
All those languages are just as dynamic as Python, more so given the dynamically loading of code with image systems, across network, with break into debugger/condition points and redo workflows.
Agreed. I'd like CPython to offer the possibility to opt in semantics that are more amenable to optimizations, similar to what Cider is enabling with their opt-in strict modules and static classes: https://github.com/facebookincubator/cinder.
> It turns out that if you have language semantics that make optimizations hard, making a fast optimizing compiler is hard. Who woulda thunk?
Is this in the article? I don't see Python's semantics mentioned anywhere as a symptom (but I only skimmed).
> shows how poorly a lot of people understand what goes into making a language fast.
...I'm sorry but are you sure you're not one of these people? Some facts:
1. JS is just as dynamic and spaghetti as Python and I hope we're all aware that it has some of the best jits out there;
2. Conversely, C++ has many "optimizing compiler[s]" and they're not all magically great by virtue of compiling a statically typed, rigid language like C++.
JIT and VM writer here. I’m also pretty clued in on how CPython works because I ported it to Fil-C.
I think if I was being paid to make CPython faster I’d spend at least a year changing how objects work internally. The object model innards are simply too heavy as it stands. Therefore, eliminating the kinds of overheads that JITs eliminate (the opcode dispatch, mainly) won’t help since that isn’t the thing the CPU spends much time on when running CPython (or so I would bet).
Many changes of that kind have been made by the faster-cpython team I believe, Mark Shannon was rather focused on it (and had a decade of experience of that kind of tweaks to python).
But I'm trying to find/recall a blog post that detailed the different steps in shrinking the CPython object struct...
If you say that's not enough, more radical changes needed, I would understand.
This article doesn't do the best job explaining the broader picture - stability has been their number one priority up to this point.
- Most of the work has just been plumbing. Int/float unboxing, smarter register allocation, free-threaded safety land in 3.15+.
- Most JIT optimizations are currently off by default or only triggers after a few thousand hits, and skips any byte-codes that look risky (profiling hooks, rare ops, etc.).
Hi, author of the post here, stability indeed has been a priority. There are some points which are not exactly the case though:
> - Most of the work has just been plumbing. Int/float unboxing, smarter register allocation, free-threaded safety land in 3.15+.
The first part is true, but for the second sentence: none of that is guaranteed to land in 3.15+. We proposed to land them, that doesn't mean they will. Landing a PR in CPython is subject to maintainer time and reviewer approval, which doesn't always happen. I proposed a few optimizations for 3.14 that never landed.
> Most JIT optimizations are currently off by default or only triggers after a few thousand hits
It is indeed true we only trigger after a few thousand hits, but all optimizations that we currently have are always enabled. We don't sandbag the JIT on purpose.
Does anyone know why for example the Ruby team is able to create JITs that are performant with comparative ease to Python? They are in many ways similar languages, but Python has 10x the developers at this point.
Ruby in both its semantics and implementation is very close to smalltalk and does not really use the Python's object model that can be summarized as "everything is a dict with string keys". That makes all the tricks discovered over last 40 years of how to make Smalltalk and Lisp fast much more directly applicable in Ruby.
My complete _guess_ (in which I make a bunch of assumptions!) is that generally it seems like the Ruby team has been more willing to make small breaking changes, whereas it seems a lot like the Python folks have become timid in those regards after the decade of transition from 2 -> 3.
Smalltalk, Self, Lisp, are highly dynamic, their JIT research are the genesis of modern JIT engines.
For some strange reason, Python community rather learns C, calls it "Python", instead of focusing why languages that are just as dynamic, have managed already a few decades ago.
What fundamentals would make the jit, this specific jit faster? Because if it's demonstrably slower, it begs the question if it can be faster or is inherently slower than a decent optimisation path through a compiler.
At this point it's a great didactic tool and a passion project surely? Or, has advantages in other dimensions like runtime size, debugging, and .pyc coverage, or in thread safe code or ...
In JavaScript, an unoptimizing JIT (no regalloc, no optimizations that look at patterns of ops, no analysis) is faster than the interpreter because it eliminates opcode dispatch.
Adding more optimizations improves things from there.
But the point is, a JIT can be a speedup just because it isn’t an interpreter (it doesn’t dynamically dispatch ops).
According to the promises of the Faster CPython Team, the JIT with a >50% speedup should have happened two years ago.
Everyone knows Python is hard to optimize, that's why Mojo also gave up on generality. These claimed 20-30% speedups, apparently made by one of the chief liars who canceled Tim Peters, are not worth it. Please leave Python alone.
Two years ago was Python 3.11, my real world workloads did see a ~15-20% improvement in performance with that release.
I don't remember the Faster CPython Team claiming JIT with a >50% speedup should have happened two years ago, can you provide a source?
I do remember Mark Shannon proposed an aggressive timeline for improving performance, but I don't remember him attributing it to a JIT, and also the Faster CPython Team didn't exist when that was proposed.
> apparently made by one of the chief liars who canceled Tim Peters
Also, I really can not think who you would be referring to as part of the Faster CPython Team, of which all the former members I am aware of largely stayed out of the discussions on DPO.
eigenspace|7 months ago
To be clear, this seems like a cool project and I dont want to be too negative about it, but i just think this was an entirely foreseeable outcome, and the amount of people excited about this JIT project when it was announced shows how poorly a lot of people understand what goes into making a language fast.
jerf|7 months ago
It was not universal. But it was very common and at least plausibly a majority view, so this idea wasn't just some tiny minority view either.
I consider this idea falsified now, pending someone actually coming up with a JIT/compiler/whatever that achieves this goal. We've poured millions upon millions of dollars into the task and the scripting languages still are not as fast as C or static languages in general. These millions were not wasted; there were real speedups worth having, even if they are somewhat hard on RAM. But they have clearly plateaued well below "C speed" and there is currently no realistic chance of that happening anytime soon.
Some people still have not noticed that the idea has been falsified and I even occasionally run into someone who thinks Javascript actually is as fast as C in general usage. But it's not and it's not going to be.
pjmlp|7 months ago
All those languages are just as dynamic as Python, more so given the dynamically loading of code with image systems, across network, with break into debugger/condition points and redo workflows.
ngrilly|7 months ago
pizlonator|7 months ago
Something else is going on.
manypineapples|7 months ago
almostgotcaught|7 months ago
Is this in the article? I don't see Python's semantics mentioned anywhere as a symptom (but I only skimmed).
> shows how poorly a lot of people understand what goes into making a language fast.
...I'm sorry but are you sure you're not one of these people? Some facts:
1. JS is just as dynamic and spaghetti as Python and I hope we're all aware that it has some of the best jits out there;
2. Conversely, C++ has many "optimizing compiler[s]" and they're not all magically great by virtue of compiling a statically typed, rigid language like C++.
pizlonator|7 months ago
I think if I was being paid to make CPython faster I’d spend at least a year changing how objects work internally. The object model innards are simply too heavy as it stands. Therefore, eliminating the kinds of overheads that JITs eliminate (the opcode dispatch, mainly) won’t help since that isn’t the thing the CPU spends much time on when running CPython (or so I would bet).
kzrdude|7 months ago
But I'm trying to find/recall a blog post that detailed the different steps in shrinking the CPython object struct...
If you say that's not enough, more radical changes needed, I would understand.
cs_throwaway|7 months ago
serjester|7 months ago
- Most of the work has just been plumbing. Int/float unboxing, smarter register allocation, free-threaded safety land in 3.15+.
- Most JIT optimizations are currently off by default or only triggers after a few thousand hits, and skips any byte-codes that look risky (profiling hooks, rare ops, etc.).
I really recommend this talk with one of the Microsoft faster Cpython developers for more details, https://www.youtube.com/watch?v=abNY_RcO-BU
kenjin4096|7 months ago
> - Most of the work has just been plumbing. Int/float unboxing, smarter register allocation, free-threaded safety land in 3.15+.
The first part is true, but for the second sentence: none of that is guaranteed to land in 3.15+. We proposed to land them, that doesn't mean they will. Landing a PR in CPython is subject to maintainer time and reviewer approval, which doesn't always happen. I proposed a few optimizations for 3.14 that never landed.
> Most JIT optimizations are currently off by default or only triggers after a few thousand hits
It is indeed true we only trigger after a few thousand hits, but all optimizations that we currently have are always enabled. We don't sandbag the JIT on purpose.
ecshafer|7 months ago
dfox|7 months ago
abhorrence|7 months ago
pjmlp|7 months ago
Smalltalk, Self, Lisp, are highly dynamic, their JIT research are the genesis of modern JIT engines.
For some strange reason, Python community rather learns C, calls it "Python", instead of focusing why languages that are just as dynamic, have managed already a few decades ago.
adgjlsfhk1|7 months ago
cuchoi|7 months ago
Seems like the development was funded by Shopify and they got a ~20% performance improvement. https://shopify.engineering/ruby-yjit-is-production-ready
A similar experience in the Python community is that Microsoft funded "Faster CPython" and they made Python 20-40% faster.
ggm|8 months ago
At this point it's a great didactic tool and a passion project surely? Or, has advantages in other dimensions like runtime size, debugging, and .pyc coverage, or in thread safe code or ...
teruakohatu|7 months ago
Unoptimised jit < optimised interpreter (at least in this instance)
They are working on it presumably because they think there will eventually be a speed ups in general or at least for certain popular workloads.
pizlonator|7 months ago
Adding more optimizations improves things from there.
But the point is, a JIT can be a speedup just because it isn’t an interpreter (it doesn’t dynamically dispatch ops).
bgwalter|7 months ago
Everyone knows Python is hard to optimize, that's why Mojo also gave up on generality. These claimed 20-30% speedups, apparently made by one of the chief liars who canceled Tim Peters, are not worth it. Please leave Python alone.
notatallshaw|7 months ago
I don't remember the Faster CPython Team claiming JIT with a >50% speedup should have happened two years ago, can you provide a source?
I do remember Mark Shannon proposed an aggressive timeline for improving performance, but I don't remember him attributing it to a JIT, and also the Faster CPython Team didn't exist when that was proposed.
> apparently made by one of the chief liars who canceled Tim Peters
Tim Peters still regularly posts on DPO so calling him "cancelled" is a choice: https://discuss.python.org/u/tim.one/activity.
Also, I really can not think who you would be referring to as part of the Faster CPython Team, of which all the former members I am aware of largely stayed out of the discussions on DPO.
unknown|7 months ago
[deleted]
throwaway032023|7 months ago
gjvc|7 months ago
pjmlp|7 months ago
unknown|7 months ago
[deleted]
firesteelrain|7 months ago