Really cool article. Posts like this always make me wonder what the state of the programming would be if browsers hadn't sucked up almost all of the world's compiler optimizers.
Look at what Mike Pall did with LuaJIT2 - I assume if the world wasn't so focused on the web, we would see more of it in other languages. But really, things aren't that bad. Microsoft has enough good people working on RyuJIT, PyPy has some of the best people advancing metatracing JITs, and Mike Pall is a god among men.
But considering that optimized scalar code performance has moved, what, maybe 40% over the last two decades, I'm going to say "not much". Compilers are sexy, but they're very much a solved problem. If we were all forced to get by with the optimized performance we saw from GCC 2.7.2, I think we'd all survive. Most of us wouldn't even notice the change.
That seems a bit chicken-and-the-egg-y though: if the web didn't become a global phenomenon then there would be far less interest in improving the browsers, far less business need for programmers, and far fewer people working on whatever they'd be working on if they weren't working on browsers.
They're solving a non-issue. The rest of the world is perfectly fine with the statically typed, well designed languages that are easy to compile. And only the web world is so obssessed with smart compilers compensating (impressively, but still far from being sufficient) for multiple deficiencies in the language design.
While they have sucked up a lot, it is far from certain all would have been employed optimizing compilers if that hadn't happened. Demand tend to create the supply.
Also, many of the improvements done for JS will certainly trickle down to Python, Ruby, PHP eventually.
Good to see the Webkit team (mostly Apple) continue putting serious energy into JS performance. Take a bit of guts to throw out the whole LLVM layer in order to get compilation performance...
It's also encouraging to see them opening up about future directions rather than just popping full-blown features from the head of Zeus every so often. (Not that they owe us anything... ;-)
(Edit: it's also damned impressive for 2 people in 3-4 months.)
tl;dr: B3 will replace LLVM in the FTL JIT of webkit. LLVM isn't performing fast enough for JIT mainly because it's so memory hungry and misses optimisations that depend on javascript semantics. They got an around 5x compile time reduction and from 0% up to around 10% performance boost in general.
>> "tl;dr: B3 will replace LLVM in the FTL JIT of webkit. LLVM isn't performing fast enough for JIT mainly because it's so memory hungry and misses optimisations that depend on javascript semantics. They got an around 5x compile time reduction and from 0% up to around 10% performance boost in general." [1]
Is this a knock on LLVM then?
I wonder then specifically if this brings to light any concerns over Swift (another dynamic language, and was created by the same person who created LLVM as well). [2]
Seems weird that the original creator of LLVM was able to make a dynamic language such as Swift - without any problems.
You're reading it too politically. It says right up in the first paragraph:
> While LLVM is an excellent optimizer, it isn’t specifically designed for the optimization challenges of dynamic languages like JavaScript.
LLVM was, more-or-less, designed around C++ semantics. B3 is not going to be a better C++ compiler than LLVM. LLVM is not going to be a better JS compiler than B3. They're taking different evolutionary paths.
Not a knock on LLVM. The conditions are just different. There is no software that is 100% great at every single thing - that's why Windows embedded CE failed while iOS and Android thrived on phone and mobile devices.
VxWorks is great at driving martian rovers, but would be horrible at running Adobe Photoshop or Maya.
Why does everything either need to be a snub or a boon?
LLVM is great at generating compiled binaries - it just takes resources (time + memory) to do so - in practical terms, the compilation time only affects the programmers and developers running multiple compiles over a work day, working on the codebase itself. For consumers who are running the programs, LLVM makes great optimized binaries.
In a JS engine, the compiler doesn't have access to the source code until a user loads up the page, so it's in essence doing "compilation" on the spot. And in this case compilation memory and time usage matters.
Swift itself is a "compiled" lanugage - meaning that in development, a developer needs to hit that "compile" button in XCode to generate a binary file which is then distributed and run. In this case, LLVM can use all cores on that big honking Mac Pro or shiny MacBook Pro 15" retina all it wants - take all 16 cpus and 32 gigs of ram and crank it and make a nice binary that comes out optimized to run on a iPad Air.
Of course things like Java conflates the two styles (source code compiled to bytecode and then to run on the JVM) - but in the Java ecosystem there are significant optimizations in both the compile phase (ie javac helloworld.java) as well as ahead of time compilation in the JVM itself.
LLVM is designed for languages that are compiled ahead of time like C and C++. In these cases it's not that important how much time and memory it takes to compile something.
If you're working on a JIT, this matters. A lot. Webkit isn't the first project has has used or worked with LLVM in this manner and abandoned it. PyPy has also investigated using LLVM and stopped doing that.
LLVM isn't bad, it just isn't good at solving this problem it wasn't designed for. If anything, I think it's remarkable LLVM has worked out as well for Webkit as it did.
Swift is generally ahead-of-time compiled, so compile speed doesn't matter as much. Many of the core parts of Swift also avoid highly dynamic semantics. The most dynamic parts are when interfacing with Objective-C classes, an area that has not been heavily optimized yet.
It's worth noticing that most of the optimizations here are for "space" -- reducing the working set size or the number of memory accesses. CPUs have gotten much faster than memory blah blah. This is the sort of thing where microbenchmarks may mislead you, because you WSS is probably not realistic.
I think we don't have great tools for helping with this sort of optimization. One can use perf to find cache misses, but that doesn't necessarily tell the whole story, as you might blame some other piece of code for causing a miss. Maybe I should try cachegrind again...
I would say that geometric mean is the usual way of averaging benchmark scores. It has the property that a given relative speedup on a component benchmark always has the same effect on the aggregate score. With an arithmetic mean the component benchmarks with a longer runtime will dominate the aggregate. Normalizing the results before applying the arithmetic mean doesn't really help either -- the first X% improvement to a component benchmark would still be valued more than the second X% speedup.
Interestingly, much of their complaints around pointer chasing, etc, are things LLVM plans on solving in the next 6-8 months. i'm a bit surprised they never bothered to email the mailing list and say "hey guys, any plans to resolve this" before going and doing all of this work. But building new JITs is fun and shiny, so ...
LLVM instruction selection is slow, there is a "fast-path" which hasn't received much attention (it is only used for -O0 in clang). The new instruction selector work just started and will take a couple of years, considering the tradeoff between spending 3 months on it and waiting a few years for LLVM to be improved (without any guarantee of LLVM reaching the same speed as what they did).
See also some thoughts from a LLVM developer on optimizing for high-level languages: http://lists.llvm.org/pipermail/llvm-dev/2016-February/09546...
I'm really wondering about the politics behind all this. I mean both LLVM and WebKit are Apple projects (even though they're Open Source). So it would have been reasonable to expect an improvement of LLVM instead of ditching it altogether.
I highly doubt there are any politics behind it. LLVM is an AOT C/C++ compiler at its core, and the tradeoffs it makes don't always make sense for dynamic, JIT compiled languages with extreme emphasis on compilation speed like JS. Personally, I expected this to happen.
Specialising LLVM for dynamic compilation might well come at the expense of LLVM's strengths as an AoT compiler, and it would probably not be easy in any case.
In addition to that, dedicated implementations can take various shortcuts to make their job easier - there are some nice examples given in the link. LuaJIT is example of a compiler project that benefits from being heavily specialised to a particular job, to remarkable effect.
munificent|10 years ago
beagle3|10 years ago
ajross|10 years ago
But considering that optimized scalar code performance has moved, what, maybe 40% over the last two decades, I'm going to say "not much". Compilers are sexy, but they're very much a solved problem. If we were all forced to get by with the optimized performance we saw from GCC 2.7.2, I think we'd all survive. Most of us wouldn't even notice the change.
mrspeaker|10 years ago
sklogic|10 years ago
yxhuvud|10 years ago
Also, many of the improvements done for JS will certainly trickle down to Python, Ruby, PHP eventually.
cpr|10 years ago
It's also encouraging to see them opening up about future directions rather than just popping full-blown features from the head of Zeus every so often. (Not that they owe us anything... ;-)
(Edit: it's also damned impressive for 2 people in 3-4 months.)
MrBuddyCasino|10 years ago
legulere|10 years ago
zepto|10 years ago
alberth|10 years ago
Is this a knock on LLVM then?
I wonder then specifically if this brings to light any concerns over Swift (another dynamic language, and was created by the same person who created LLVM as well). [2]
Seems weird that the original creator of LLVM was able to make a dynamic language such as Swift - without any problems.
[1] https://news.ycombinator.com/item?id=11105231
[2] http://nondot.org/sabre/
chipsy|10 years ago
> While LLVM is an excellent optimizer, it isn’t specifically designed for the optimization challenges of dynamic languages like JavaScript.
LLVM was, more-or-less, designed around C++ semantics. B3 is not going to be a better C++ compiler than LLVM. LLVM is not going to be a better JS compiler than B3. They're taking different evolutionary paths.
barkingcat|10 years ago
VxWorks is great at driving martian rovers, but would be horrible at running Adobe Photoshop or Maya.
Why does everything either need to be a snub or a boon?
LLVM is great at generating compiled binaries - it just takes resources (time + memory) to do so - in practical terms, the compilation time only affects the programmers and developers running multiple compiles over a work day, working on the codebase itself. For consumers who are running the programs, LLVM makes great optimized binaries.
In a JS engine, the compiler doesn't have access to the source code until a user loads up the page, so it's in essence doing "compilation" on the spot. And in this case compilation memory and time usage matters.
Swift itself is a "compiled" lanugage - meaning that in development, a developer needs to hit that "compile" button in XCode to generate a binary file which is then distributed and run. In this case, LLVM can use all cores on that big honking Mac Pro or shiny MacBook Pro 15" retina all it wants - take all 16 cpus and 32 gigs of ram and crank it and make a nice binary that comes out optimized to run on a iPad Air.
Of course things like Java conflates the two styles (source code compiled to bytecode and then to run on the JVM) - but in the Java ecosystem there are significant optimizations in both the compile phase (ie javac helloworld.java) as well as ahead of time compilation in the JVM itself.
DasIch|10 years ago
If you're working on a JIT, this matters. A lot. Webkit isn't the first project has has used or worked with LLVM in this manner and abandoned it. PyPy has also investigated using LLVM and stopped doing that.
LLVM isn't bad, it just isn't good at solving this problem it wasn't designed for. If anything, I think it's remarkable LLVM has worked out as well for Webkit as it did.
om2|10 years ago
pizlonator|10 years ago
jlebar|10 years ago
I think we don't have great tools for helping with this sort of optimization. One can use perf to find cache misses, but that doesn't necessarily tell the whole story, as you might blame some other piece of code for causing a miss. Maybe I should try cachegrind again...
panic|10 years ago
jsnell|10 years ago
DannyBee|10 years ago
Joky|10 years ago
Alphasite_|10 years ago
unknown|10 years ago
[deleted]
ck2|10 years ago
https://webkit.org/blog-files/kraken.png
https://webkit.org/blog-files/octane.png
seriously though, dang, how many years of coding to get to that level of expertise
Ecco|10 years ago
pcwalton|10 years ago
gsg|10 years ago
In addition to that, dedicated implementations can take various shortcuts to make their job easier - there are some nice examples given in the link. LuaJIT is example of a compiler project that benefits from being heavily specialised to a particular job, to remarkable effect.
zepto|10 years ago