It's somehow never the tech companies' fault for willfully designing inept feedback channels or even null-routed feedback channels in Google's case to impede customers communicating with them. I think many companies, especially given $200b in savings, could have handled this report better. Many companies without $200b can receive information from a customer without it passing through journalists first.
What's especially pathetic is it doesn't matter what you're reporting - a grave security bug, a widespread hardware flaw, a longing for better functionality - Apple doesn't want to know. In fact they warned iOS developers against trying to get their attention.
If you run to the press and trash us, it never helps.
That Medium article presents the quotation as though it's literally something that Apple wrote. It doesn't actually appear to on the page it supposedly came from, nor anything like it, as far as I can see.
I can only imagine the amount of bug reports, real and false, that a company of Apple's size must receive on a daily basis. Is there any company at that scale that can reliably filter through all of them to find actual, critical bugs quickly?
It simply isn't as easy as saying 'flag all reports with 'security vulnerability' in the submission for priority.' That could still be thousands of reports in the 'priority' queue, most of which some person would need to manually investigate one by one.
If you are able to perform the following steps for any of Amazon, Google, Facebook, Netflix, Microsoft or Twitter, I will literally eat a hat (you may choose what kind):
1. Discover an easily exploitable vulnerability that allows access to a chosen user's private data
2. Email their security address about it
3. Tweet at them about it
4. Fax them about it
5. A week later the vulnerability is still exploitable
You do not have to play fair. You're allowed to impersonate a suburban mom or a grandpa who's not good with technology. You're allowed to ramble or use vague and non-technical terms as long as a reasonably qualified person could determine what the vulnerability is. Specifically, it is not required that you include relevant product versions, steps to reproduce, a full reproduction video, or a one-sentence impact summary like "a caller can eavesdrop on the recipient of a Group Facetime call without their knowledge or consent".
There's no apologising this away. The vulnerability was already a monumental fuckup, but this detail propels it into the realm of cultural dysfunction. It should not be possible to fail this badly. If you put listening devices in people's pockets, you need to hold yourself to a higher standard than "I dunno, bug reporting is hard".
Apple has a dedicated team that triages incoming security vulnerability reports. If you search for how to report a security issue to Apple, you would find their e-mail address.
The weird thing to me is that the NYT article says that she tried "faxing Apple’s security team." Having some familiarity with the team and the process of reporting security vulnerabilities to them, I do not recall them ever claiming to have a fax machine. The idea of Apple asking you to fax in a bug report is ludicrous.
There are a couple odd gaps that seem to be been in place at Apple.
First, Apple did not even acknowledge the report to the initial finder. This is flawed, like Google, Microsoft and other big players acknowledge receipt.
Secondly, the person tried to go beyond after not hearing anything, by calling, faxing and other means to no avail.
Third, Apple needs to staff their responders better it seems. Most issues can be filtered out quickly to be left with the few interesting ones. The repro here doesnt need any technical knowledge!
Many companies outsource the initial triage process even to have it scale when needed.
So, one take away for Apple is to improve their response process and transparency. Transparency they have always lacked when it comes to security.
Also when a user reports a major flaw in your product and your response is that you will not lift a finger until the user fills the correct form, you have reached the state of a useless bureaucracy, completely unconcerned about the quality of your product. I remember being given that advice by an employee on the vendor’s own forum where I reported a problem. They are still waiting for their bug report and I have found an alternative product.
There's a guy on my team (of 12 people) in Apple that literally does nothing but screen bugs. I think he screened 100 bugs last week and there are still hundreds more unscreened. It's really hard to keep track of.
Raymond Chen (at Microsoft), for example has written repeatedly about incidents he spent lots of time investigating where your first instinct is "That's not a bug" and the eventual outcome was "Yup, that's not a bug" but it was a security ticket and so Raymond doggedly chased down every aspect to make sure it isn't a problem.
He mostly invokes the Hitchiker's Guide quote "It rather involved being on the other side of this airtight hatchway" to suggest that often the problem in these bugs is that they have a step where you've legitimately got privileges, and then they use those privileges to... do something you legitimately need privileges for. That's not a bug, if you try it without the privileges it doesn't work, but Raymond has to walk through all the steps seeing whether anything surprising is going on.
Now, whether engineers at a corp actually get given the space to do this stuff is an executive policy decision. Maybe at Apple not enough of them do, I can't say. But if it doesn't get done you are sooner or later going to miss cases where it _sounds_ like it's not a bug but actually there's a serious bug if you looked closely.
If this were their only QA/QC issue, then fine, it might be excusable.
But it's this, broken MBP keyboards, broken MBP hinges, bent iPads, the MBP core-i9 thermal issue and probably a few I'm forgetting.
Nintendo doesn't have these kinds of failures, they're pretty analogous to Apple. Hardware and software. What's the difference? Nintendo cares and Apple doesn't. (Not to say Nintendo is issue free by any means).
Maybe they should put some of that $250b in cash to work upgrading their processes.
It's not hard to reach a living person who works at Apple. The next step should have been reproducing it for that person. It's not clear that the finders in this story did that - it seems they might have been trying to find out about bounty payments first.
If any tech support or Apple Genius were to have seen this bug reproduced, it should have immediately been easy to flag to the right person.
Years ago my team and I discovered a pretty significant bug in Safari's/CFNetworking's TLS implementation. Once the browser had deemed a certificate valid once, it would subsequently accept it for all hostnames. We got absolutely nowhere with Apple's official security contacts. The issue only got resolved months later, after I was able to find an employee from their security team at WWDC and explain the issue face to face.
Care to tell how it went? Did he have an expanation why the process was so crappy? Did hebmaybe even knew about your bug report but was unable tonfonsomething sbout it because of some beaurocracy?
When I saw the headline, I assumed it was a situation where someone had emailed the wrong address or only tried to contact them via Twitter. But upon reading the article I see this is a high-quality report. She was sounding alarms and emailing all the right people. It's is insane that Apple missed this.
I think at this point, we need Tim Cook to write an apology piece about how they screwed up, how this won't happen again, and who got fired.
We also need some kind of hardware indicator like a light that indicates when the mic or camera are turned on. After a blunder like this, a privacy-focused needs to rebuild trust that they take privacy seriously.
Not turning out to be a great week for Apple. Even if they do receive a large number of bug reports, I would like to think they have the resources (let's face it, they're not cash-strapped) to resolve something as critical and privacy-focused as this. Their failure to do so makes a mockery of their users who pay a significant premium for their products, often in the name of privacy.
What is happening with Apple - people used to justify the high cost of Apple devices claiming they paid for the "high quality". But now ... First the "bug" that allowed root access on macOS and now this "bug" that literally allowed anyone to spy on you through your iPhone? Not to speak of iPads / iPhones that bend, ios throttling due to weak batteries etc. etc.
I'm no Steve Jobs fan or Apple apologist but it seems obvious enough - Jobs was the force driving the company forward and now the momentum he created is finally starting to wear off. I really hope they can get their shit together sooner or later as they still seem the best of a bad bunch wrt privacy - at least for now.
I submitted it before those 2 articles, but it got lost on new. It got resubmitted by the admin, who sent me this email:
We thought you might like to know that we put https://news.ycombinator.com/item?id=19029573
in the second-chance pool, so it will get a random placement on the front page
sometime in the next 24 hours.
This is part of an experiment in giving good HN submissions multiple
chances at the front page. If you're curious, you can read about it at
https://news.ycombinator.com/item?id=11662380 and other links there.
People who thinks that what happened is unacceptable needs to understand that Apple must receive a lot of these types of call every week. What would you do if someone send you multiple messages saying that they found a major issue _without even detailling anything_ while this person actually wants you to give them money for what they found (that they still haven't disclosed any information about it)? I'm sure the majority would ignore these calls unless some details were shared about the issue.
I am not surprised about what happened at all. There is an argument that can be made about the fact that it took Apple so many years to finally implement group video call that they could take a little bit of time to do it right but other than that, I don't see how Apple could have prevented a bug that a person wasn't willing to disclose without having money first.
If they aren't hiring enough people to clear their bug report queue every day, that's still unacceptable.. They're sitting on gigantic piles of cash. They can afford it.
Another product stream and company (and hype) I was never a fan of. Best thing they did was to rip off FreeBSD and the worst was break *nix compliant userspace + influence design UX and UI patterns for a new generation.
There has been recently some activity here in HN regarding formal model checking and protocol verification (TLA+, SPIN, Promela...) I guess they are relevant to this case.
Maybe I'm just too old and contankerious and just don't "get it" but warning Apple via Twitter[0] isn't really following a Coordinated Vulnerability Disclosure process, yeah?
Reading that website you provided, the mother was correct to hit up twitter as they suggest that "Customers" contact Apple Support. Their twitter is an official channel to that end.
The only area in that document she would have followed to reach out Apple's security contact is under the heading of "Security and privacy researchers", of which I am doubting she thought herself or her 14 year old son as.
The tweet wasn't the warning. The tweet describes the 14 year old's mom sending a formal notice of some kind.
Anyway, it's absurd to say thay tweets aren't enough warning if they're public. A public disclosure may be distasteful to Apple, but it doesn't change the fact that they have a security emergency.
> isn't really following a Coordinated Vulnerability Disclosure process, yeah
Why should they have followed that process? Coordinated Vulnerability Disclosure is just one way of many of disclosing security problems. It's not the single right way.
Apple staff should copy/paste the information reported to them on Twitter to the relevant team for triage or investigation. The user is already helping enough by telling them on Twitter.
(according to the article) They didn't spend over a week developing a fix to the bug, it took them over a week to just disable the feature (given the severity of the bug this should have been done as soon as they knew about it).
It also sounds like the person reporting the bug was ignored until she made a developer account and reported the bug through that - that shouldn't have been necessary.
The fix itself depends on what the problem is - clearly they went for a “disable server side” route, but I’m sure they also tried to work out if they could filter on the server side. My guess is they can’t because the “call accepted” response is probably in the encrypted data stream.
Rolling a system update is also not immediate (eg you need to be sure that you don’t result in user flows resulting in no audio), hence the focus on server side fixes (that lead to today’s “kill server side”) fix.
But also “a week to fix” depends - it’s a week from the report, at which point it needs to be screened. So let’s say a day to get to an actual engineer - it may take longer to get appropriate security keywords attached - then the engineer would need to actually read it, which depends on what their current workload is, where they are in their release schedule, etc.
It’s not trivial - all big companies get thousands of big reports a day, and they just take time to get to the right place.
I agree with what other have said (telling a consumer to get an ADC account is clearly suboptimal). But in general the path from consumer to security big is challenging for everyone.
[+] [-] benologist|7 years ago|reply
What's especially pathetic is it doesn't matter what you're reporting - a grave security bug, a widespread hardware flaw, a longing for better functionality - Apple doesn't want to know. In fact they warned iOS developers against trying to get their attention.
https://medium.com/@krave/apple-s-app-store-review-process-i...[+] [-] tome|7 years ago|reply
https://developer.apple.com/app-store/review/guidelines/
[EDIT: Apple indeed literally write that in a previous version of the page. Wow.]
[+] [-] lozenge|7 years ago|reply
[+] [-] jm20|7 years ago|reply
It simply isn't as easy as saying 'flag all reports with 'security vulnerability' in the submission for priority.' That could still be thousands of reports in the 'priority' queue, most of which some person would need to manually investigate one by one.
[+] [-] sgentle|7 years ago|reply
1. Discover an easily exploitable vulnerability that allows access to a chosen user's private data
2. Email their security address about it
3. Tweet at them about it
4. Fax them about it
5. A week later the vulnerability is still exploitable
You do not have to play fair. You're allowed to impersonate a suburban mom or a grandpa who's not good with technology. You're allowed to ramble or use vague and non-technical terms as long as a reasonably qualified person could determine what the vulnerability is. Specifically, it is not required that you include relevant product versions, steps to reproduce, a full reproduction video, or a one-sentence impact summary like "a caller can eavesdrop on the recipient of a Group Facetime call without their knowledge or consent".
There's no apologising this away. The vulnerability was already a monumental fuckup, but this detail propels it into the realm of cultural dysfunction. It should not be possible to fail this badly. If you put listening devices in people's pockets, you need to hold yourself to a higher standard than "I dunno, bug reporting is hard".
[+] [-] rgovostes|7 years ago|reply
The weird thing to me is that the NYT article says that she tried "faxing Apple’s security team." Having some familiarity with the team and the process of reporting security vulnerabilities to them, I do not recall them ever claiming to have a fax machine. The idea of Apple asking you to fax in a bug report is ludicrous.
[+] [-] kerng|7 years ago|reply
First, Apple did not even acknowledge the report to the initial finder. This is flawed, like Google, Microsoft and other big players acknowledge receipt.
Secondly, the person tried to go beyond after not hearing anything, by calling, faxing and other means to no avail.
Third, Apple needs to staff their responders better it seems. Most issues can be filtered out quickly to be left with the few interesting ones. The repro here doesnt need any technical knowledge!
Many companies outsource the initial triage process even to have it scale when needed.
So, one take away for Apple is to improve their response process and transparency. Transparency they have always lacked when it comes to security.
[+] [-] cm2187|7 years ago|reply
[+] [-] why_only_15|7 years ago|reply
[+] [-] tialaramex|7 years ago|reply
He mostly invokes the Hitchiker's Guide quote "It rather involved being on the other side of this airtight hatchway" to suggest that often the problem in these bugs is that they have a step where you've legitimately got privileges, and then they use those privileges to... do something you legitimately need privileges for. That's not a bug, if you try it without the privileges it doesn't work, but Raymond has to walk through all the steps seeing whether anything surprising is going on.
Now, whether engineers at a corp actually get given the space to do this stuff is an executive policy decision. Maybe at Apple not enough of them do, I can't say. But if it doesn't get done you are sooner or later going to miss cases where it _sounds_ like it's not a bug but actually there's a serious bug if you looked closely.
[+] [-] bleriot|7 years ago|reply
[+] [-] philip1209|7 years ago|reply
[+] [-] jssmith|7 years ago|reply
[+] [-] huffmsa|7 years ago|reply
But it's this, broken MBP keyboards, broken MBP hinges, bent iPads, the MBP core-i9 thermal issue and probably a few I'm forgetting.
Nintendo doesn't have these kinds of failures, they're pretty analogous to Apple. Hardware and software. What's the difference? Nintendo cares and Apple doesn't. (Not to say Nintendo is issue free by any means).
Maybe they should put some of that $250b in cash to work upgrading their processes.
[+] [-] unknown|7 years ago|reply
[deleted]
[+] [-] jonathanberger|7 years ago|reply
If any tech support or Apple Genius were to have seen this bug reproduced, it should have immediately been easy to flag to the right person.
[+] [-] markonen|7 years ago|reply
[+] [-] jackson1way|7 years ago|reply
[+] [-] FactolSarin|7 years ago|reply
I think at this point, we need Tim Cook to write an apology piece about how they screwed up, how this won't happen again, and who got fired.
[+] [-] jdavis703|7 years ago|reply
[+] [-] osrec|7 years ago|reply
[+] [-] webmobdev|7 years ago|reply
Something is quite wrong ...
[+] [-] SmellyGeekBoy|7 years ago|reply
[+] [-] dang|7 years ago|reply
[+] [-] josu|7 years ago|reply
We thought you might like to know that we put https://news.ycombinator.com/item?id=19029573 in the second-chance pool, so it will get a random placement on the front page sometime in the next 24 hours.
This is part of an experiment in giving good HN submissions multiple chances at the front page. If you're curious, you can read about it at https://news.ycombinator.com/item?id=11662380 and other links there.
[+] [-] trumped|7 years ago|reply
[+] [-] ninedays|7 years ago|reply
I am not surprised about what happened at all. There is an argument that can be made about the fact that it took Apple so many years to finally implement group video call that they could take a little bit of time to do it right but other than that, I don't see how Apple could have prevented a bug that a person wasn't willing to disclose without having money first.
[+] [-] LfLxfxxLxfxx|7 years ago|reply
You don't see this kind of stuff every week, and surely Apple has the resources to at least confirm it.
[+] [-] empath75|7 years ago|reply
[+] [-] romeisendcoming|7 years ago|reply
[+] [-] pantulis|7 years ago|reply
This stuff is hard.
[+] [-] mishurov|7 years ago|reply
[deleted]
[+] [-] qrbLPHiKpiux|7 years ago|reply
[+] [-] auiya|7 years ago|reply
[+] [-] renholder|7 years ago|reply
[0] - https://resources.sei.cmu.edu/asset_files/SpecialReport/2017...
EDIT: Changed the link to the CERT guide for CVD.
[+] [-] ibero|7 years ago|reply
The only area in that document she would have followed to reach out Apple's security contact is under the heading of "Security and privacy researchers", of which I am doubting she thought herself or her 14 year old son as.
[+] [-] smt88|7 years ago|reply
Anyway, it's absurd to say thay tweets aren't enough warning if they're public. A public disclosure may be distasteful to Apple, but it doesn't change the fact that they have a security emergency.
[+] [-] chrisseaton|7 years ago|reply
Why should they have followed that process? Coordinated Vulnerability Disclosure is just one way of many of disclosing security problems. It's not the single right way.
[+] [-] benologist|7 years ago|reply
[+] [-] a-dub|7 years ago|reply
[+] [-] jachee|7 years ago|reply
[+] [-] p1necone|7 years ago|reply
It also sounds like the person reporting the bug was ignored until she made a developer account and reported the bug through that - that shouldn't have been necessary.
[+] [-] olliej|7 years ago|reply
Rolling a system update is also not immediate (eg you need to be sure that you don’t result in user flows resulting in no audio), hence the focus on server side fixes (that lead to today’s “kill server side”) fix.
But also “a week to fix” depends - it’s a week from the report, at which point it needs to be screened. So let’s say a day to get to an actual engineer - it may take longer to get appropriate security keywords attached - then the engineer would need to actually read it, which depends on what their current workload is, where they are in their release schedule, etc.
It’s not trivial - all big companies get thousands of big reports a day, and they just take time to get to the right place.
I agree with what other have said (telling a consumer to get an ADC account is clearly suboptimal). But in general the path from consumer to security big is challenging for everyone.
[+] [-] briandear|7 years ago|reply