Let's ignore the rather awkward self promotion, and the fact that 2FA would have prevented this specific incident.
This is the important part, which everyone should think about:
> What would have been way worse — immeasurably worse — is if our team had used Slack for anything other than what we did use it for, which was discussing outages of our own product. Had my cofounder and I discussed our company's cap table, or business partnerships, or compensation agreements, or ongoing legal matters over Slack; or had our team traded API keys, or security-sensitive matters; or had we controlled mission-critical infrastructure via Slack-powered "bots"; we'd be sweating bullets to this day that our important company secrets were out in the open, about to resurface at the worst possible time.
I see Slack being used for everything at a lot of companies.
There are a lot of interesting things in the chat history everywhere: SSH/API keys, logins and passwords, sensitive internal documents, chatops bots that would let one take control of the infrastructure, and worst of all - juicy office affair gossip.
Combine this with often lax user management (forgetting to disable old accounts, inviting people to channels they shouldn't be in, ...).
Most companies overlook this and don't even have a policy for what's OK to post on Slack, and what isn't. Not even to dream of any kind of enforcement.
I'ts a big security problem, even without Slack getting compromised, and should be on the radar for CTOs.
At my last job, I wrote a little tool to download hipchat chat logs and scrape them for anything that looks like a password. There were tons!! I raised it to the CTO/CIO, and while they sounded interested in it, they never enacted any policies, etc, to prevent the sort of password sharing that anyone can apparently easily take advantage of.
In fact, the CIO asked me to send him a spreadsheet of when/where passwords were shared -- and then never did anything with it, despite the file essentially being a password map of the infrastructure. When I asked him if he deleted it, he told me he couldn't because the company's lawyers wouldn't allow him to. Lol.
Slack has some options to delete all messages over N days old. Unless there is a really good reason not to, turning on this feature generally sounds like a good idea. At least you can drastically limit the length of the archive available to any attacker.
Do Keybase folks use Keybase for literally all their communications, document sharing? That's a bit hard to imagine. I'd wager that some of those important comms were on email and pretty much all the scary arguments about server/account hacking would apply to the involved email accounts too?
And a naive interpretation of the posting also raises the question of what secret deals, etc could Keybase be doing that would be disastrous if it were to come out later, that, as a Keybase user, I should be concerned about? A secret backdoor in the keys embedded for the NSA?
Ive been considering building a product that would help eradicate "lax user management" of Slack, and like platforms. It would basically keep an audit trail of managers approval for their specific team members they manage access to specific platforms and products. Basically, helping scale that one IT guy whom should somehow know every person at the company in detail.
If I succeeded in making the process easy & less time consuming than it is now, do you think it's a product you would pay for?
> or had our team traded API keys, or security-sensitive matters; or had we controlled mission-critical infrastructure via Slack-powered "bots"
That's why I always strongly recommend having a https://privatebin.info/ . In fact, in every startup I have been, I setup an instance of that forcing "burn after reading" and as a policy all credentials and sensitive info goes there when sharing.
The author would have done well by refraining from using this as an opportunity to make a sales pitch for their startup, as it detracts from an otherwise important message.
Let me see if I have this right:
Slack had a major security breach in 2015. Apparently someone installed malicious code that could even read password inputs in plaintext. They waited 4 years, after growing large and going public, to inform affected users. And in the interim they blamed their users for any related security problems.
Do I have this correct? If so, how is anyone going to defend this situation? And how can anyone put any sensitive data on Slack, or tell their company to do so, and feel good about it now?
I expected some stupid apology note from the CEO on their website if this turns into a bigger issue, which is sort of an anti-pattern at this point...
The issue looks to be that they thought they had informed all the affected users back in 2015, but underestimated the set of affected users. The breach certainly wasn’t secret until today, they posted it publicly at that time: https://slackhq.com/march-2015-security-incident-and-the-lau....
Would shareholders have a case to make for fraud here? Slack clearly didn't want this information getting out pre-IPO, as a security disclosure in this case would certainly impact public confidence in the company.
This is the "lite" version of his sales pitch. The real sales pitch was the email that Keybase users just received. It said, basically, that Slack was compromised but if you would have used Keybase instead, you wouldn't have been compromised.
Awkward, indeed.
I cringe when I see companies try to use a competitor's misfortune to their advantage.
What makes this even more sad is the extreme difficulty you'll have if you attempt to remove your company data off of the Slack platform. (Disclaimer: I stopped using Slack 2 years ago)
Our company used Slack extensively for multiple years. A couple years ago, we decided to stop using Slack for official company communication. After switching to alternative communication tools, we tried to delete the data in our Slack account and found it to be nearly impossible.
I recall using 3rd party python scripts (opensource on Github) that took hours to run - the script used an API key to fetch and delete messages individually.
We also tried using Slack's Admin panel to delete messages. At the time, I believe it required clicking a checkbox next to every chat message we wanted to delete. Clearly not a realistic way to scrub an account.
The sad reality is that with many services like Slack, once a provider has your data, there's often no easy way to remove it. IMO, this is a major downfall of our reliance on modern SaaS services (companies have no incentive to prioritize features for deleting account data - the only users who would find those features useful are already churned customers).
I have used it in conjunction with Slack; a channel for infrastructure team members to share sensitive details, or to direct message secrets to people. It works well for that light usage.
I cannot imagine replacing Slack with it for all company or team communication though. It's not remotely polished enough.
I use Keybase. I like it, the thing to keep in mind though is 99.9% of the time the biggest threat vector to your company is going to be preventing your employees from sexually harassing each other, not preventing the Russians or whoever from reading your internal messages.
Keybase has a Teams feature so it's built more towards your use-case. I use it to trade Stellar (XLM), file storage/management, and sharing/proving all of my platform identities. I haven't had a chance to get too social on it yet so I can't comment on the chatting experience, but overall it's a neat platform. It also has a command-line interface.
I’ve been running my 100% remote team on it the last few months, it works very well for us. Never been a fan of Slack, and never use any bots, so just comms.
The post says "If the attackers inject server code, 2FA or U2F or any Web-based security practice does little." Understandably if the attacker was actively scraping passwords with 2FAs and was using the credentials immediately, that would be an issue.
However, for the issue mentioned in the post, wouldn't 2FA have saved him? Supposedly, the compromised credential is from 2015.
U2F saves you, most of the other options potentially make it worse.
If the bad guys are dumb and just were there to passively steal passwords then almost anything works. But if the bad guys are paying attention and stealing authentication secrets then it breaks down like this:
Passwords: Stolen if used while bad guys could watch
TOTP/MOTP/ similar software one time codes: Seed gets stolen by bad guys, they can generate the same codes as you now.
SecurID/ similar key fob type one time codes: Seed gets stolen by bad guys, they can generate the same codes as you now.
SMS: Now bad guys know your phone number. In a targeted attack they will "jack" the number and cause even more havoc, but you might be safe against script kiddies
U2F (or WebAuthn): The server doesn't know any secrets. It knows a cookie (meaningless gibberish except to one FIDO device) and a public key (public). Stealing these is futile and cannot be used to impersonate the user.
TOTP and SecurID surprise people because they forget that although the credential transmitted is just a one-time code, the _server_ (which bad guys broke into here) needs to know the same seed as the user does to get the same codes.
Not only does Keybase not automatically update its client, there is no way to even figure out if your client is out of date and in need of security updates. Even if you look up the exact version of your installed client, which you can find, there is nothing on the website that says what the most recent version is. The only way to even get a hint is to look on GitHub, and even that isn't accurate; version 4.2.1 is the latest release, but when I download the mac app it's still version 4.2.0.
For whatever problems Slack has, at least I know if there is a new version that I need to install.
Keybase developer here. Keybase does automatically update on Mac, and you can check if you're out of date on the CLI with `keybase update check`. Additionally, the "widget" popup from the system tray will display an out of date banner automatically.
4.2.1 was a bugfix release that only affected Linux and BSD, so we didn't push out Mac or Windows releases for it.
You know what's better than installing slack? Not installing it and using it in the browser.
If there's a website option for any tool, I recommend using that over native. It's usually more performant and is less of a security risk. And it's always up to date.
That is why you use a system with a package manager. Maybe you could follow the releases feed on Github. It was what I do when I maintain a package on AUR.
.. or exploited/hacked a random cheap server to use to proxy their requests and have access to high speed 100Mbit/1Gbit downloads/uploads of Slack data.
slack in 2015: "We have no indication that the hackers were able to decrypt stored passwords, as Slack uses a one-way encryption technique called hashing."
slack in 2019: "In 2015, (...) The attackers also inserted code that allowed them to capture plaintext passwords as they were entered by users at the time. "
This reflects poorly on them. Unless they discovered only now that the 2015 hack included capturing plaintext passwords.
Encryption for a business chat app limits the potential users rather significantly, as I understand it. A number of sectors (like banking) have strict rules which require keeping a record of company communications. How does Keybase deal with this, or do they choose not to play in that market?
Well, the bank could still record from their end of the conversation, capturing the chat logs from the employee's terminal and shipping them somewhere.
That's a nice sales pitch! A little on the nose, but I think that's ok.
I think one issue keybase still had is it's minimal web presence that sounds to focus too little on what keybase can do for regular users. People need more explaining of the day to day benefits.
I wonder if this affected companies like IBM... I know for a fact IBM uses slack to talk internally about client billing, meaning if IBM was compromised the attacker knows what most of IBM clients bought from IBM.
> Keybase messages are end-to-end encrypted, and only our users control their decryption keys.
End-to-end encryption isn't just good for the users because the service can't access the messages and sell them, it's also good for the users as it provides good protection if the service gets hacked: message content can't leak unless the attackers can change client code.
As people are discussing Keybase for teams and whatnot - could anyone comment on Keybase for individuals, families, etc?
My family are debating moving to Matrix (and away from iMessage). I had briefly debates Keybase due to some interesting features. Anyone have experience with Keybase for families and individuals?
It's OK, notifications collapsing is somewhat wonky for some people and some of my friends complain about the app's performance vs something like Facebook but given the security model it will never be as fast.
The second I read this, I immediately signed up to Keybase today and deleted my Slack account and switched to the Keybase chat system instead which I am setting up right now.
I am confident to say that this incident was the final nail in the coffin for me to abandon Slack. Choosing Keybase was a no brainer.
> In other words, if you’re one of the approximately 99% who joined Slack after March 2015 or changed your password since then, this announcement does not apply to you.
Hackers compromising plaintext password would seem to apply to everyone using Slack, whether their account was compromised or not??
I’m a little confused here. Isn’t Keybase a slack replacement product? Why is the CEO of Keybase using Slack for any company communication instead of his own service? Am I missing something obvious here?
Slack gates a lot of its exfiltration security features behind enterprise licensing. Expect to double your spend if you want to try to prevent sensitive data floating around in slack.
Does Keybase support chat message search yet?
Otherwise claiming to support all important features of Slack is stretching the truth. I use search extensively at work.
> We basically just use a #breaking channel in there in case we have Keybase downtime.
The problem wasn't they were using Foo and Foo was compromised.
The problem was that he couldn't be 100% sure whether Foo was compromised or not (when asked, they claimed they weren't).
Now, if Foo is a "random-cheezburger-meme-generator" startup you just try out with a throw-away password, then you can probably assume that it's likely they have been hacked and they are lying to you.
That leaves one alternative scenario: you have been hacked.
You'd expect that Slack (in 2019), while it's still possible they lie to you, has decent security practices and thus makes it less likely to be hacked and thus increasing the odds that you might indeed be hacked.
In a way, it acted as a honeypot. But one that you don't control so you can't be sure whether it caught a hacker or whether the bees are drunk.
the_duke|6 years ago
This is the important part, which everyone should think about:
> What would have been way worse — immeasurably worse — is if our team had used Slack for anything other than what we did use it for, which was discussing outages of our own product. Had my cofounder and I discussed our company's cap table, or business partnerships, or compensation agreements, or ongoing legal matters over Slack; or had our team traded API keys, or security-sensitive matters; or had we controlled mission-critical infrastructure via Slack-powered "bots"; we'd be sweating bullets to this day that our important company secrets were out in the open, about to resurface at the worst possible time.
I see Slack being used for everything at a lot of companies.
There are a lot of interesting things in the chat history everywhere: SSH/API keys, logins and passwords, sensitive internal documents, chatops bots that would let one take control of the infrastructure, and worst of all - juicy office affair gossip.
Combine this with often lax user management (forgetting to disable old accounts, inviting people to channels they shouldn't be in, ...).
Most companies overlook this and don't even have a policy for what's OK to post on Slack, and what isn't. Not even to dream of any kind of enforcement.
I'ts a big security problem, even without Slack getting compromised, and should be on the radar for CTOs.
FabHK|6 years ago
Is that so? The article states: "If the attackers inject server code, 2FA or U2F or any Web-based security practice does little."
bpchaps|6 years ago
In fact, the CIO asked me to send him a spreadsheet of when/where passwords were shared -- and then never did anything with it, despite the file essentially being a password map of the infrastructure. When I asked him if he deleted it, he told me he couldn't because the company's lawyers wouldn't allow him to. Lol.
It's insane out there.
harryh|6 years ago
sriku|6 years ago
And a naive interpretation of the posting also raises the question of what secret deals, etc could Keybase be doing that would be disastrous if it were to come out later, that, as a Keybase user, I should be concerned about? A secret backdoor in the keys embedded for the NSA?
(Keybase user here)
johnmarcus|6 years ago
If I succeeded in making the process easy & less time consuming than it is now, do you think it's a product you would pay for?
xtracto|6 years ago
That's why I always strongly recommend having a https://privatebin.info/ . In fact, in every startup I have been, I setup an instance of that forcing "burn after reading" and as a policy all credentials and sensitive info goes there when sharing.
numair|6 years ago
Let me see if I have this right:
Slack had a major security breach in 2015. Apparently someone installed malicious code that could even read password inputs in plaintext. They waited 4 years, after growing large and going public, to inform affected users. And in the interim they blamed their users for any related security problems.
Do I have this correct? If so, how is anyone going to defend this situation? And how can anyone put any sensitive data on Slack, or tell their company to do so, and feel good about it now?
I expected some stupid apology note from the CEO on their website if this turns into a bigger issue, which is sort of an anti-pattern at this point...
johncolanduoni|6 years ago
mfer|6 years ago
save_ferris|6 years ago
chrissnell|6 years ago
Awkward, indeed.
I cringe when I see companies try to use a competitor's misfortune to their advantage.
unknown|6 years ago
[deleted]
cj|6 years ago
Our company used Slack extensively for multiple years. A couple years ago, we decided to stop using Slack for official company communication. After switching to alternative communication tools, we tried to delete the data in our Slack account and found it to be nearly impossible.
I recall using 3rd party python scripts (opensource on Github) that took hours to run - the script used an API key to fetch and delete messages individually.
We also tried using Slack's Admin panel to delete messages. At the time, I believe it required clicking a checkbox next to every chat message we wanted to delete. Clearly not a realistic way to scrub an account.
The sad reality is that with many services like Slack, once a provider has your data, there's often no easy way to remove it. IMO, this is a major downfall of our reliance on modern SaaS services (companies have no incentive to prioritize features for deleting account data - the only users who would find those features useful are already churned customers).
gtirloni|6 years ago
https://get.slack.help/hc/en-us/articles/204067366-Delete-a-...
pingec|6 years ago
privateSFacct|6 years ago
What’s super bad here is slack misleading about the cause wasting all the users time.
Quick question, anyone use key base - can u give a quick review? Team currently use slack
skuhn|6 years ago
I cannot imagine replacing Slack with it for all company or team communication though. It's not remotely polished enough.
Alex3917|6 years ago
sucrose|6 years ago
If you end up signing-up, follow me @ https://keybase.io/jbales
nabeards|6 years ago
rch|6 years ago
bluk|6 years ago
However, for the issue mentioned in the post, wouldn't 2FA have saved him? Supposedly, the compromised credential is from 2015.
tialaramex|6 years ago
If the bad guys are dumb and just were there to passively steal passwords then almost anything works. But if the bad guys are paying attention and stealing authentication secrets then it breaks down like this:
Passwords: Stolen if used while bad guys could watch
TOTP/MOTP/ similar software one time codes: Seed gets stolen by bad guys, they can generate the same codes as you now.
SecurID/ similar key fob type one time codes: Seed gets stolen by bad guys, they can generate the same codes as you now.
SMS: Now bad guys know your phone number. In a targeted attack they will "jack" the number and cause even more havoc, but you might be safe against script kiddies
U2F (or WebAuthn): The server doesn't know any secrets. It knows a cookie (meaningless gibberish except to one FIDO device) and a public key (public). Stealing these is futile and cannot be used to impersonate the user.
TOTP and SecurID surprise people because they forget that although the credential transmitted is just a one-time code, the _server_ (which bad guys broke into here) needs to know the same seed as the user does to get the same codes.
Alex3917|6 years ago
For whatever problems Slack has, at least I know if there is a new version that I need to install.
kbModalduality|6 years ago
4.2.1 was a bugfix release that only affected Linux and BSD, so we didn't push out Mac or Windows releases for it.
ashelmire|6 years ago
If there's a website option for any tool, I recommend using that over native. It's usually more performant and is less of a security risk. And it's always up to date.
j605|6 years ago
valar_m|6 years ago
Reminds me of the time Amazon learned that using S3 to host status indicators for S3 is a bad idea if S3 goes down.
zuck9|6 years ago
The author might know this already but the hackers aren't Dutch. They simply bought a cheap NL server from LeaseWeb to mask their real IP.
nacs|6 years ago
.. or exploited/hacked a random cheap server to use to proxy their requests and have access to high speed 100Mbit/1Gbit downloads/uploads of Slack data.
lurker458|6 years ago
slack in 2019: "In 2015, (...) The attackers also inserted code that allowed them to capture plaintext passwords as they were entered by users at the time. "
This reflects poorly on them. Unless they discovered only now that the 2015 hack included capturing plaintext passwords.
dlgeek|6 years ago
ekc|6 years ago
Poor security practices are poor security practices despite conflicts of interests, and Slack's are certainly extremely poor.
StreamBright|6 years ago
emdowling|6 years ago
icebraining|6 years ago
lallysingh|6 years ago
I think one issue keybase still had is it's minimal web presence that sounds to focus too little on what keybase can do for regular users. People need more explaining of the day to day benefits.
unknown|6 years ago
[deleted]
speeder|6 years ago
Same applies to a bunch of other companies...
unknown|6 years ago
[deleted]
paulcole|6 years ago
How does Keybase make money?
anchpop|6 years ago
est31|6 years ago
End-to-end encryption isn't just good for the users because the service can't access the messages and sell them, it's also good for the users as it provides good protection if the service gets hacked: message content can't leak unless the attackers can change client code.
asdkhadsj|6 years ago
My family are debating moving to Matrix (and away from iMessage). I had briefly debates Keybase due to some interesting features. Anyone have experience with Keybase for families and individuals?
FabHK|6 years ago
pzduniak|6 years ago
disclaimer: I work for Keybase
madspindel|6 years ago
rvz|6 years ago
I am confident to say that this incident was the final nail in the coffin for me to abandon Slack. Choosing Keybase was a no brainer.
Finster|6 years ago
> In other words, if you’re one of the approximately 99% who joined Slack after March 2015 or changed your password since then, this announcement does not apply to you.
Hackers compromising plaintext password would seem to apply to everyone using Slack, whether their account was compromised or not??
lorenzsell|6 years ago
oconnor663|6 years ago
> what we did use it for, which was discussing outages of our own product
You can't use Keybase to communicate about why Keybase is down :)
mnutt|6 years ago
Boulth|6 years ago
workerthread|6 years ago
nemoniac|6 years ago
gpjanik|6 years ago
Nelkins|6 years ago
tshanmu|6 years ago
RyanShook|6 years ago
ithkuil|6 years ago
The problem wasn't they were using Foo and Foo was compromised.
The problem was that he couldn't be 100% sure whether Foo was compromised or not (when asked, they claimed they weren't).
Now, if Foo is a "random-cheezburger-meme-generator" startup you just try out with a throw-away password, then you can probably assume that it's likely they have been hacked and they are lying to you.
That leaves one alternative scenario: you have been hacked.
You'd expect that Slack (in 2019), while it's still possible they lie to you, has decent security practices and thus makes it less likely to be hacked and thus increasing the odds that you might indeed be hacked.
In a way, it acted as a honeypot. But one that you don't control so you can't be sure whether it caught a hacker or whether the bees are drunk.
thecatspaw|6 years ago