The most striking thing here is that Linus has apparently dismissed incompetence as a rational explanation. Yes, he is often brash, but usually he is accusing someone of sheer stupidity. He does not do that here. Linus alleges that we are being lied to - that we don’t know the full story, nor Intel’s motives.
Furthermore, we are left to wonder if Microsoft is also being fed “bullshit” patches, and if they may be less discerning than Linus regarding a proper solution.
He's quite clear about his theory as to Intel's motives:
> The whole IBRS_ALL feature to me very clearly says "Intel is not
serious about this, we'll have a ugly hack that will be so expensive
that we don't want to enable it by default, because that would look
bad in benchmarks".
> So instead they try to push the garbage down to us. And they are doing
it entirely wrong, even from a technical standpoint."
The root question is what else is Intel trying to cover up with these garbage patches? Are they afraid of power leakage across gates allowing an attacker to gain a higher level of privilege in certain generations of silicon, and trying to cover it up with these patches (hence some of the seemingly crazy things they do)?
Linus calls people stupid when they are usually being stupid, at least in his eyes. He doesn't wantonly accuse people of stupidity for no reason. It's just that when he perceives stupid activity... well, he generally goes off. It's his main trigger.
Wonder if perhaps Intel is under classified and gagged duress from the government? There has been plenty of evidence the government is not acting with citizen security foremost in its technical and telecommunications policies.
Hanlon's razor should include an exception where PR and politics are involved. Discounting malicious people as just stupid is the reason so many "stupid" people are in power.
>The most striking thing here is that Linus has apparently dismissed incompetence as a rational explanation. Yes, he is often brash, but usually he is accusing someone of sheer stupidity. He does not do that here. Linus alleges that we are being lied to - that we don’t know the full story, nor Intel’s motives.
"And that's actually ignoring the much _worse_ issue, namely that the
whole hardware interface is literally mis-designed by morons."
Maybe you missed this line? Some classic Linus right there...
A lot of people trying to defend being a jerk as necessary in these circumstances. I think Google culture proves the opposite: Googley code and peer reviews, blameless postmortems, and a host of other mechanisms that de-escalate and de-personalize things.
The Meltdown work at Google probably didn't begin with a rant about morons.
It's possible to construct a culture where people can air grievances and criticize others without inducing flame wars. It only works for Linus because people won't go nuclear in the response, but for rank and file engineers, especially of equal stature, if you call someone a moron in a thread, it's likely to kill productivity and create a negative testosterone ladden atmosphere of people trying to avoid being wrong, and counter attacking others.
I've worked at a lot of places where engineer got into heated shouting matches. It's is not a way to increase the probability of zeroing in on a problem, or reaching agreement faster.
I must say that I'm really happy that Linux is taking a stance on this one. He doesn't care what the legal consequences to Intel are, he is pushing for a proper technical solution damned be the consequences.
And here[1] is Woodhouse's coherent latest reply as of a couple hours ago on the issue, which explains for all us in the peanut gallery (his words, which I think are spot on) exactly what this is about, why it's included, how it actually affects the situation, why it was put forth at all, etc.
It appears much less sinister than Linus was insinuating, but Linus has yet to reply.
That's an interesting summary but I predict that Linus will destroy him for completely avoiding the hard question: why the fuck IBRS_ALL would not be on by default on future "fixed" chips, if on such CPU it would not be somehow crappy.
And the risk of it becoming architectural (with that absurd default) is insane too.
In case it’s unclear to anyone, David Woodhouse (the person Linus is replying to / previous poster) works for Amazon and previously Intel[1] aka @dwmw2 on Twitter, ironically his twitter profile is: “Kernel hacker. Known to occasionally promote an attitude of violence towards complete morons.”
*[1] Correction, I thought (as does Google) that he still worked at Intel, but it was pointed out that he now works at Amazon UK.
It worries me that this isn't higher both here and on Reddit. I read through the rest of the email thread this morning and wanted to say something but I'm worried that I don't understand it well enough.
I always appreciate Linus' rants. In addition to being somewhat humorous, I usually learn a thing or two from them. It amazes me that people focus on how much of a jerk he's being instead of actually looking at the contents of his emails and learning something.
Really? The internet is full of ranting commenters. They all have content of a sort. His don't deserve any more attention just because he's the subject of a hero-worship cult.
Its actually possible to find thoughtful commentary on most any subject, without suffering jerks.
I have a gut feeling that all of this madness is driven by legal and management terror. Doing the right thing likely means, to many people, admitting fault. Legal has likely banned anyone from doing anything that looks like an admission of guilt. Management and PR, etc. are all probably about saving face, too. The people who care about doing the right thing from a technology perspective likely don't have any power.
I think we should count ourselves lucky we have someone so good at spotting bullsh*t like this - and isn’t afraid to call it out as they sees it.
Intel & friends have absolutely disgusted me lately, it could be all to easy to settle for less based on their standards, but let’s not let them set the standard - they’ve shown they can’t be trusted with quality and when they fail - they can’t be trusted to be transparent or even take ownership of the problems they create.
Not only spot, but also invests the energy in calling it out and explaining it.
Many business failures i've witnessed happened ultimately because nobody had the energy to cut down the bullshit forest that constantly regrows around a company.
From what I gather, some of this is cultural. His country of origin is just generally blunt and direct about things.
That directness is generally associated with multicultural environments where subtler context-dependent communication simply does not work. Some of the quotes I am seeing here from him are things I think are unnecessarily harsh, but most are not.
Discussions about something he has said or written seem to inevitably comment on his speaking style and divide up between those who praise it and those who vilify it. I feel it is mostly a positive, but discussions could benefit from a little context and broader perspective of that detail.
I also see Intel lawyers working overtime, closely interacting with marketing, to somehow ward of lawsuits coming from Google, Microsoft, Amazon, which must be seeking ways out of Intel contracts to turn towards AMD processors for their currently more attractive TCO characteristics.
Actually that's pretty scary. Are the rest of heavy weight kernel code maintainers too stupid for than then? If it weren't for Linus' hard judgement? Would we then buying badly engineered products without ever questioning?
I think you are very harsh with "they’ve shown they can’t be trusted with quality". Their product is incredibly complex, it's amazing only so few mistakes like this happen.
I totally agree with "they can’t be trusted to be transparent or even take ownership of the problems they create" though. When you play at Intel's level and mess up on that order of magnitude, doing what they did make you loose all trust.
> Intel & friends have absolutely disgusted me lately [...] - they can’t be trusted to be transparent or even take ownership of the problems they create.
Well, it's the natural outcome of a system based entirely on competition. When the only incentive is "beat everyone else, nothing else matters" then this eventually happens.
There's lots of people who call out bullshit. But people usually don't listen or respect them unless they have power - which Linus has. There are plenty of other professionals who have been complaining lately.
It does make me worry about the day that Linus retires. I imagine the kernel has enough momentum to carry on for a while but without a leader like Linus I can see it very slowly starting to unravel and quality issues starting to sneak in, much like we're seeing with post-Jobs Apple.
Thankfully it doesn't seem that time is coming any time soon.
Are people reading the rest of the thread? Linus admitted mistaking an acronym and changed his tone from "pushing garbage for unclear reasons" to just "garbage patches".
Can it be that Linus is so angry with people and their incompetence because he actually gives a shit about Linux and the code?
I'm not interested if his rants are ethic or not, if they are effective or not. I'm just trying to understand why he's ranting? Because he's an ill adjusted person? Or is there something else?
Linus is 'infamous' for being very angry with people who write shitty code. Why is that? If you read the chapters written by him in 'The Hacker Ethic', you find out that he codes because he's enjoying it, it's what makes him happy, so to speak. Therefore, he is invested in the code he writes. It's not just a job for him. It's his life's project, his 'baby' if you will.
Can it be that because he's so emotionally invested into the code, when he sees that other people push shitty code and don't care about it, the intensity of the emotions he feels are way higher than the emotions of a manager who's just 'doing his job'? Can it be that because he cares so much, he's having a hard time not reacting so 'emotionally'?
I think we've covered the technical part of this now, not that you like
it â not that any of us *like* it. But since the peanut gallery is
paying lots of attention it's probably worth explaining it a little
more for their benefit.
This is all about Spectre variant 2, where the CPU can be tricked into
mispredicting the target of an indirect branch. And I'm specifically
looking at what we can do on *current* hardware, where we're limited to
the hacks they can manage to add in the microcode.
The new microcode from Intel and AMD adds three new features.
One new feature (IBPB) is a complete barrier for branch prediction.
After frobbing this, no branch targets learned earlier are going to be
used. It's kind of expensive (order of magnitude ~4000 cycles).
The second (STIBP) protects a hyperthread sibling from following branch
predictions which were learned on another sibling. You *might* want
this when running unrelated processes in userspace, for example. Or
different VM guests running on HT siblings.
The third feature (IBRS) is more complicated. It's designed to be
set when you enter a more privileged execution mode (i.e. the kernel).
It prevents branch targets learned in a less-privileged execution mode,
BEFORE IT WAS MOST RECENTLY SET, from taking effect. But it's not just
a 'set-and-forget' feature, it also has barrier-like semantics and
needs to be set on *each* entry into the kernel (from userspace or a VM
guest). It's *also* expensive. And a vile hack, but for a while it was
the only option we had.
Even with IBRS, the CPU cannot tell the difference between different
userspace processes, and between different VM guests. So in addition to
IBRS to protect the kernel, we need the full IBPB barrier on context
switch and vmexit. And maybe STIBP while they're running.
Then along came Paul with the cunning plan of "oh, indirect branches
can be exploited? Screw it, let's not have any of *those* then", which
is retpoline. And it's a *lot* faster than frobbing IBRS on every entry
into the kernel. It's a massive performance win.
So now we *mostly* don't need IBRS. We build with retpoline, use IBPB
on context switches/vmexit (which is in the first part of this patch
series before IBRS is added), and we're safe. We even refactored the
patch series to put retpoline first.
But wait, why did I say "mostly"? Well, not everyone has a retpoline
compiler yet... but OK, screw them; they need to update.
Then there's Skylake, and that generation of CPU cores. For complicated
reasons they actually end up being vulnerable not just on indirect
branches, but also on a 'ret' in some circumstances (such as 16+ CALLs
in a deep chain).
The IBRS solution, ugly though it is, did address that. Retpoline
doesn't. There are patches being floated to detect and prevent deep
stacks, and deal with some of the other special cases that bite on SKL,
but those are icky too. And in fact IBRS performance isn't anywhere
near as bad on this generation of CPUs as it is on earlier CPUs
*anyway*, which makes it not quite so insane to *contemplate* using it
as Intel proposed.
That's why my initial idea, as implemented in this RFC patchset, was to
stick with IBRS on Skylake, and use retpoline everywhere else. I'll
give you "garbage patches", but they weren't being "just mindlessly
sent around". If we're going to drop IBRS support and accept the
caveats, then let's do it as a conscious decision having seen what it
would look like, not just drop it quietly because poor Davey is too
scared that Linus might shout at him again. :)
I have seen *hand-wavy* analyses of the Skylake thing that mean I'm not
actually lying awake at night fretting about it, but nothing concrete
that really says it's OK.
If you view retpoline as a performance optimisation, which is how it
first arrived, then it's rather unconventional to say "well, it only
opens a *little* bit of a security hole but it does go nice and fast so
let's do it".
But fine, I'm content with ditching the use of IBRS to protect the
kernel, and I'm not even surprised. There's a *reason* we put it last
in the series, as both the most contentious and most dispensable part.
I'd be *happier* with a coherent analysis showing Skylake is still OK,
but hey-ho, screw Skylake.
The early part of the series adds the new feature bits and detects when
it can turn KPTI off on non-Meltdown-vulnerable Intel CPUs, and also
supports the IBPB barrier that we need to make retpoline complete. That
much I think we definitely *do* want. There have been a bunch of us
working on this behind the scenes; one of us will probably post that
bit in the next day or so.
I think we also want to expose IBRS to VM guests, even if we don't use
it ourselves. Because Windows guests (and RHEL guests; yay!) do use it.
If we can be done with the shouty part, I'd actually quite like to have
a sensible discussion about when, if ever, we do IBPB on context switch
(ptraceability and dumpable have both been suggested) and when, if
ever, we set STIPB in userspace.
Are state actors or others trying to take advantage of meltdown needing fixes & patches to insert their own version of fixes & patches that end up benefiting them?
> As a hack for existing CPUs, it's just about tolerable â as long as it
> can die entirely by the next generation.
> Certainly it's a nasty hack, but hey â the world was on fire and in the
> end we didn't have to just turn the datacentres off and go back to goat
> farming, so it's not all bad.
Off-topic, but what is with the "â" in these sentences? Is that just some weird encoding error, or does it actually have meaning?
Probably an encoding error. An m-dash is E2 80 94 in UTF-8. (An n-dash is E2 80 93.) I suspect that the original UTF-8 is misinterpreted as ISO 8859-1 or a similar encoding, where 0xE2 is “â”, and the other two are unassigned or control characters.
It's upsetting that the media and some of the mainstream audience see Linus as a "crank" that only exists to complain and tear people down. Examples like this show how valuable his experience is. I would NEVER ask him to water himself down knowing it would likely make his technical analysis worse.
[+] [-] bashcoder|8 years ago|reply
Furthermore, we are left to wonder if Microsoft is also being fed “bullshit” patches, and if they may be less discerning than Linus regarding a proper solution.
[+] [-] azernik|8 years ago|reply
> The whole IBRS_ALL feature to me very clearly says "Intel is not serious about this, we'll have a ugly hack that will be so expensive that we don't want to enable it by default, because that would look bad in benchmarks".
> So instead they try to push the garbage down to us. And they are doing it entirely wrong, even from a technical standpoint."
[+] [-] StudentStuff|8 years ago|reply
[+] [-] icelancer|8 years ago|reply
[+] [-] rabboRubble|8 years ago|reply
[+] [-] jasonkostempski|8 years ago|reply
[+] [-] aphextron|8 years ago|reply
"And that's actually ignoring the much _worse_ issue, namely that the whole hardware interface is literally mis-designed by morons."
Maybe you missed this line? Some classic Linus right there...
[+] [-] cromwellian|8 years ago|reply
The Meltdown work at Google probably didn't begin with a rant about morons.
It's possible to construct a culture where people can air grievances and criticize others without inducing flame wars. It only works for Linus because people won't go nuclear in the response, but for rank and file engineers, especially of equal stature, if you call someone a moron in a thread, it's likely to kill productivity and create a negative testosterone ladden atmosphere of people trying to avoid being wrong, and counter attacking others.
I've worked at a lot of places where engineer got into heated shouting matches. It's is not a way to increase the probability of zeroing in on a problem, or reaching agreement faster.
[+] [-] dotancohen|8 years ago|reply
[+] [-] smcleod|8 years ago|reply
[+] [-] walterstucco|8 years ago|reply
[deleted]
[+] [-] kbenson|8 years ago|reply
It appears much less sinister than Linus was insinuating, but Linus has yet to reply.
1: http://lkml.iu.edu/hypermail/linux/kernel/1801.2/05282.html
[+] [-] wilun|8 years ago|reply
[+] [-] smcleod|8 years ago|reply
*[1] Correction, I thought (as does Google) that he still worked at Intel, but it was pointed out that he now works at Amazon UK.
[+] [-] trynewideas|8 years ago|reply
[+] [-] koheripbal|8 years ago|reply
[+] [-] swiley|8 years ago|reply
[+] [-] PrimHelios|8 years ago|reply
[+] [-] JoeAltmaier|8 years ago|reply
Its actually possible to find thoughtful commentary on most any subject, without suffering jerks.
[+] [-] Animats|8 years ago|reply
[+] [-] Waterluvian|8 years ago|reply
[+] [-] mrmondo|8 years ago|reply
Intel & friends have absolutely disgusted me lately, it could be all to easy to settle for less based on their standards, but let’s not let them set the standard - they’ve shown they can’t be trusted with quality and when they fail - they can’t be trusted to be transparent or even take ownership of the problems they create.
[+] [-] Roritharr|8 years ago|reply
Many business failures i've witnessed happened ultimately because nobody had the energy to cut down the bullshit forest that constantly regrows around a company.
[+] [-] DoreenMichele|8 years ago|reply
That directness is generally associated with multicultural environments where subtler context-dependent communication simply does not work. Some of the quotes I am seeing here from him are things I think are unnecessarily harsh, but most are not.
Discussions about something he has said or written seem to inevitably comment on his speaking style and divide up between those who praise it and those who vilify it. I feel it is mostly a positive, but discussions could benefit from a little context and broader perspective of that detail.
[+] [-] jhoechtl|8 years ago|reply
I also see Intel lawyers working overtime, closely interacting with marketing, to somehow ward of lawsuits coming from Google, Microsoft, Amazon, which must be seeking ways out of Intel contracts to turn towards AMD processors for their currently more attractive TCO characteristics.
Could be the end of Intel as we know it.
[+] [-] integricho|8 years ago|reply
[+] [-] dschuetz|8 years ago|reply
[+] [-] sametmax|8 years ago|reply
I totally agree with "they can’t be trusted to be transparent or even take ownership of the problems they create" though. When you play at Intel's level and mess up on that order of magnitude, doing what they did make you loose all trust.
[+] [-] Random_Person|8 years ago|reply
[+] [-] Florin_Andrei|8 years ago|reply
Well, it's the natural outcome of a system based entirely on competition. When the only incentive is "beat everyone else, nothing else matters" then this eventually happens.
[+] [-] ashelmire|8 years ago|reply
[+] [-] mpweiher|8 years ago|reply
[deleted]
[+] [-] delta1|8 years ago|reply
This made me so nervous on his behalf
[+] [-] johnflan|8 years ago|reply
[+] [-] ageofwant|8 years ago|reply
Lord knows where we would be today if we had leaders like this more frequently through history.
[+] [-] SmellyGeekBoy|8 years ago|reply
Thankfully it doesn't seem that time is coming any time soon.
[+] [-] eru|8 years ago|reply
[deleted]
[+] [-] shoover|8 years ago|reply
[+] [-] SpEd3Y|8 years ago|reply
Can it be that Linus is so angry with people and their incompetence because he actually gives a shit about Linux and the code?
I'm not interested if his rants are ethic or not, if they are effective or not. I'm just trying to understand why he's ranting? Because he's an ill adjusted person? Or is there something else?
Linus is 'infamous' for being very angry with people who write shitty code. Why is that? If you read the chapters written by him in 'The Hacker Ethic', you find out that he codes because he's enjoying it, it's what makes him happy, so to speak. Therefore, he is invested in the code he writes. It's not just a job for him. It's his life's project, his 'baby' if you will.
Can it be that because he's so emotionally invested into the code, when he sees that other people push shitty code and don't care about it, the intensity of the emotions he feels are way higher than the emotions of a manager who's just 'doing his job'? Can it be that because he cares so much, he's having a hard time not reacting so 'emotionally'?
[+] [-] Aissen|8 years ago|reply
http://lkml.iu.edu/hypermail/linux/kernel/1801.2/05282.html
[+] [-] fouc|8 years ago|reply
[+] [-] ratinacage|8 years ago|reply
> Certainly it's a nasty hack, but hey â the world was on fire and in the > end we didn't have to just turn the datacentres off and go back to goat > farming, so it's not all bad.
Off-topic, but what is with the "â" in these sentences? Is that just some weird encoding error, or does it actually have meaning?
[+] [-] DerekL|8 years ago|reply
[+] [-] ksk|8 years ago|reply
[+] [-] bertolo1988|8 years ago|reply
[+] [-] krisives|8 years ago|reply