> According to police, the worker had initially suspected he had received a phishing email from the company’s UK office, as it specified the need for a secret transaction to be carried out. However, the worker put aside his doubts after the video call because other people in attendance had looked and sounded just like colleagues he recognized.
Regardless of the sophistication of the deepfake, surely this rings huge alarm bells, right? I'm not even sure I'd be comfortable making secret transactions on instruction from my boss. Even if your boss is actually asking you to do this, how can you have the financial authority to transfer $25M and not the savvy to think that being asked to transfer huge amounts of money in secret isn't going to result in you getting thrown under the bus?
The scammers usually have a slightly unusual, but plausible (and urgent) story.
For example, that they've just closed a deal to buy a startup - a negotiation which was of course conducted in secrecy. It's a startup in another country, which is why we're all out of the office. Timezones are why you've received the request outside of normal working hours. And we've got to, um, close the deal so we can announce it outside of stock market opening hours, for both countries. To close the deal we've got to pay 10% of the 250M purchase price upfront. If you can't get this done within 2 hours the deal will fall through.
Secret doesn't mean illegal. Unless something is illegal, this guy doesn't have any input and it's up to the auditor to verify the legitimacy of the transactions.
> Regardless of the sophistication of the deepfake, surely this rings huge alarm bells, right?
Its a company with revenues of a couple of billion and that probably sub contracts thousands of other companies on projects around the world. The finance department is probably sending similar payments regularly.
Most payments will be "secret" in that the amounts won't be made public to employees that don't need to know. The company maybe, for example, be repeating work that has been already been done in house so doesn't want it known inhouse what companies are being paid.
I've heard this happening for a local company with about £9 million with a similar email scam. Supposedly, the person who transferred the money was competent and clever. With that amount of money, saying you don't need to ask questions, in part, is very convincing.
Yeah, this would at least cause me to email my boss and say "Can you just confirm, you want me to transfer $25 million to this account? I'll hold off until you give me confirmation in writing"
hell i do this if our tester hasn't managed to go over some aspect of our release. That way i get in writing from the product owner that he has OKd it, and if he sends me a teams message i ask him to email me confirmation.
If you are a boomer company that does not know how online works, then you can also afford a boomer-style business class flight tickets to do a secret $25M transaction face-to-face.
Electronic engineers spent decades overcoming thermal noise floors so that humans could communicate over vast distances with small amounts of energy.
AI researchers, in a few short years, undid all that by making computer-generated chatter and images indistinguishable from messages sent by humans.
Until such a time as we live in a Bladerunner-like world of Replicants, being in-person will be the only reliable way to convey a message from human to human.
I'm long on travel and in-person meetings, short on VR and telecoms.
When I'm on a company video call, the people I'm meeting with are logged into their company accounts, through the fancy company authentication system. Large warnings are displayed if there are any external participants, and I wouldn't be surprised if it's possible to disable the ability to even have guests. Third-party video conference software is banned and blocked from installation on work computers.
I am not in the finance department, but in software engineering and operations, two-party controls are everywhere. I can't check in code without reviews. I can't access production systems or make changes without approval from another team member. I would think that similar processes could be put into place for transferring tens of millions of dollars.
In other words, there are ways to deal with this that don't come down to "mistrust all technology and revert to face-to-face meetings and handing cash to each other".
Isn't it also possible to scam people in in person meetings by pretending to be someone you aren't? The new thing with deepfakes is that you can pretend to be someone that the victim knows.
What you said only applies to communication that requires authenticating a party. Most cases that doesn't matter. Voice or video communication used to be inherently unfakeable, now that it is fakeable, we'll just treat it like text comms, relying on secure channels, signing etc.
Human written text has been indistinguishable from machine written text for a very long time. We've still managed to maintain chains of trust to discern legitimate messages with decent success rates.
> making computer-generated chatter and images indistinguishable from messages sent by humans.
That's a false dichotomy. "computer-generated chatter and images" ARE messages sent by humans. There are no cases of computers having agency known to me yet. The root of the problem is humans who lie and mislead. Now they merely have more avenues to do so. In the same vein, you could blame the electronic engineers for allowing people to lie quickly and over vast distances.
We need new ident protocol just for AI. I think that's part of Altman doing that orb thingy with iris scanner. It's creepy though and I'll never touch that things.
I mean - we have authentication for bank accounts, why wouldn't that be demanded for transactions like this? Without proper authentication of the authorities there's no way that a transaction like this should be put through.
I know it's a meme and all, but doesn't blockchain solve this? Ok, Mr. Guy who looks like my boss on Video Call, I can send those funds, just sign the transaction with your private key and it'll all be done.
What I find strange about this is you dont need it to be "deepfake".
Just an inside job.
If a large company allows a single employee to transfer millions to a new bank account/vendor that has no history, on "their belief" the instruction came from an approved person (i.e. their boss, CFO etc) - that company has major governance issues that are not related to deepfake.
Imagine the more simple scenario - an employee transfers millions, knowingly fraudulantly, to some people they are working with. They then simply supply some "deep fake" pictures and a story how it was an accident - and boom; you walk away with millions.
Checks and balances exist for many reasons - deepfake doesnt overcome those by itself. This company is just missing basic steps that would have protected itself here.
edit: in fact- its even more obviously some inside job; put the deepfake aside for a moment. How was the meeting even booked? Their PR person said "none of our internal systems were compromised". So this meeting magically appeared in someone's calendar? Using their internal video system (Skype or Teams or whatever). And the criminals knew to target this person, with enough knowledge of random office people to deep fake them? Come on...
You're the only commenter using critical analysis, everyone else is just flapping their jaws.
I hate discussing deepfakes. I'm one of the original patent holders of automated actor replacement technology. I developed it for personalized advertising, after having been an actor replacement specialist in a bunch of VFX film you probably saw.
I spent from 2002 to '08 creating a VFX pipeline, with global patent protections, and an ethical guidance that included public education on this fundamental new technology. Long story short, I needed financing, went to VCs and angels and they were perfectly winning to fund a porn company, but not what I'd planned: an ethical rollout of a sensitive and very powerful technology with many legs, few realize even today.
By '13 I was bankrupt, burned out, and one of my tech partners, a global leader in facial recognition hired me. That's a different story. Actor replacement technology is a fundamental capability with applications far more important than fraud and pornography. But our civilization is far far too immature to realize any of them.
The real issue here is a lack of proper risk controls around business processes involving money. Regardless of if it’s £3 for a coffee or £25m for a Secret acquisition there should be an agreed process that everyone involved in business transactions should be aware of so that if they are suddenly privy to a deal they can navigate and validate the authenticity of their involvement.
This brings up an interesting “risk control” that one of my tech investors personally implemented with his family, in case a audio/video version of him ever asks to do anything crazy: secret passwords, agreed upon in person.
How cool is it that Zoom is capturing our data and using it to train their "AI" efforts? Perhaps nobody in the world is better positioned to completely disrupt nearly every tech-using company, by emulating our C-suites, and ordering us to drain everything. Imagine the Robin Hood shenanigans they could get up to! Or the evil supervillain shenanigans! Who cares, really. The future is so fun!
I think the underlying problem is cultural: people have been conditioned to expect others to authenticate them (give me the last 4 digits of your SSN, tell me the last two transactions on your account), BUT they haven't been told they need to authenticate others. They just aren't thinking "how do I know this is who I think it is? How do I know they haven't been kidnapped?".
My personal protocol with my bank when they ring me is for me to call them back.
The bank workers are normally quite understanding - except when it is someone from fraud detection (and yes these are legitimate calls) and they tend to get odly defensive that I wont hand out my personal information.
I think one way to verify legitimacy is by calling back. For example, "Ok boss, just protocol, but you wouldn't mind if I call you using your usual number?" (or just do it without informing in advance).
Ideally they screen-record (can you do that in Android/iPhone?), so at least if it's really scam, they can say "but I follow protocol, here's the evidence".
Btw we once had a similar scam attempt. "The CEO" emailed Finance in great urgency to transfer money. Good thing the CEO was sitting next to the Finance lady. I was sitting next to them watching the horror turned comedy.
Learned recently my mid sized company was also targeted. The CFO received first a (fake) call from a lawyer asking to confirm a transaction, then later a deep faked voicemail from the co founder mentioning that same transaction. It apparently all sounded very real. The attacks are becoming very targeted, customized and elaborate. Very far from the Nigerian prince emails written in poor English...
The article lacks details to discuss this particular incident. This could reasonably be a company with poor governance, insecure configuration and authentication - and then this is a non-story. OTOH, with the amount of money in question, a sophisticated attack is absolutely believeable, and even 2FA and better process governance will help you out. Maybe a PKI does, but as always, it depends.
Explain your reasoning for this. I work in the office, but the vast (98%) of my meetings are on teams or zoom. When you work in a company with multiple locations (and in different countries) working in our assigned office isn’t going to help at all.
bartlettD|1 year ago
Regardless of the sophistication of the deepfake, surely this rings huge alarm bells, right? I'm not even sure I'd be comfortable making secret transactions on instruction from my boss. Even if your boss is actually asking you to do this, how can you have the financial authority to transfer $25M and not the savvy to think that being asked to transfer huge amounts of money in secret isn't going to result in you getting thrown under the bus?
michaelt|1 year ago
For example, that they've just closed a deal to buy a startup - a negotiation which was of course conducted in secrecy. It's a startup in another country, which is why we're all out of the office. Timezones are why you've received the request outside of normal working hours. And we've got to, um, close the deal so we can announce it outside of stock market opening hours, for both countries. To close the deal we've got to pay 10% of the 250M purchase price upfront. If you can't get this done within 2 hours the deal will fall through.
csomar|1 year ago
llamaimperative|1 year ago
vintermann|1 year ago
helsinkiandrew|1 year ago
Its a company with revenues of a couple of billion and that probably sub contracts thousands of other companies on projects around the world. The finance department is probably sending similar payments regularly.
Most payments will be "secret" in that the amounts won't be made public to employees that don't need to know. The company maybe, for example, be repeating work that has been already been done in house so doesn't want it known inhouse what companies are being paid.
markogrady|1 year ago
el_oni|1 year ago
hell i do this if our tester hasn't managed to go over some aspect of our release. That way i get in writing from the product owner that he has OKd it, and if he sends me a teams message i ask him to email me confirmation.
miohtama|1 year ago
tompccs|1 year ago
Electronic engineers spent decades overcoming thermal noise floors so that humans could communicate over vast distances with small amounts of energy.
AI researchers, in a few short years, undid all that by making computer-generated chatter and images indistinguishable from messages sent by humans.
Until such a time as we live in a Bladerunner-like world of Replicants, being in-person will be the only reliable way to convey a message from human to human.
I'm long on travel and in-person meetings, short on VR and telecoms.
masto|1 year ago
I am not in the finance department, but in software engineering and operations, two-party controls are everywhere. I can't check in code without reviews. I can't access production systems or make changes without approval from another team member. I would think that similar processes could be put into place for transferring tens of millions of dollars.
In other words, there are ways to deal with this that don't come down to "mistrust all technology and revert to face-to-face meetings and handing cash to each other".
changoplatanero|1 year ago
dmos62|1 year ago
spacebanana7|1 year ago
persnickety|1 year ago
That's a false dichotomy. "computer-generated chatter and images" ARE messages sent by humans. There are no cases of computers having agency known to me yet. The root of the problem is humans who lie and mislead. Now they merely have more avenues to do so. In the same vein, you could blame the electronic engineers for allowing people to lie quickly and over vast distances.
jessetemp|1 year ago
mrkramer|1 year ago
noAnswer|1 year ago
blitzo|1 year ago
sgt101|1 year ago
I mean - we have authentication for bank accounts, why wouldn't that be demanded for transactions like this? Without proper authentication of the authorities there's no way that a transaction like this should be put through.
ta93754829|1 year ago
csomar|1 year ago
imgabe|1 year ago
laurencei|1 year ago
Just an inside job.
If a large company allows a single employee to transfer millions to a new bank account/vendor that has no history, on "their belief" the instruction came from an approved person (i.e. their boss, CFO etc) - that company has major governance issues that are not related to deepfake.
Imagine the more simple scenario - an employee transfers millions, knowingly fraudulantly, to some people they are working with. They then simply supply some "deep fake" pictures and a story how it was an accident - and boom; you walk away with millions.
Checks and balances exist for many reasons - deepfake doesnt overcome those by itself. This company is just missing basic steps that would have protected itself here.
edit: in fact- its even more obviously some inside job; put the deepfake aside for a moment. How was the meeting even booked? Their PR person said "none of our internal systems were compromised". So this meeting magically appeared in someone's calendar? Using their internal video system (Skype or Teams or whatever). And the criminals knew to target this person, with enough knowledge of random office people to deep fake them? Come on...
skanderbm|1 year ago
bsenftner|1 year ago
I hate discussing deepfakes. I'm one of the original patent holders of automated actor replacement technology. I developed it for personalized advertising, after having been an actor replacement specialist in a bunch of VFX film you probably saw.
I spent from 2002 to '08 creating a VFX pipeline, with global patent protections, and an ethical guidance that included public education on this fundamental new technology. Long story short, I needed financing, went to VCs and angels and they were perfectly winning to fund a porn company, but not what I'd planned: an ethical rollout of a sensitive and very powerful technology with many legs, few realize even today.
By '13 I was bankrupt, burned out, and one of my tech partners, a global leader in facial recognition hired me. That's a different story. Actor replacement technology is a fundamental capability with applications far more important than fraud and pornography. But our civilization is far far too immature to realize any of them.
illwrks|1 year ago
legel|1 year ago
graemep|1 year ago
klyrs|1 year ago
dboreham|1 year ago
blitzar|1 year ago
The bank workers are normally quite understanding - except when it is someone from fraud detection (and yes these are legitimate calls) and they tend to get odly defensive that I wont hand out my personal information.
wiradikusuma|1 year ago
Ideally they screen-record (can you do that in Android/iPhone?), so at least if it's really scam, they can say "but I follow protocol, here's the evidence".
Btw we once had a similar scam attempt. "The CEO" emailed Finance in great urgency to transfer money. Good thing the CEO was sitting next to the Finance lady. I was sitting next to them watching the horror turned comedy.
EZ-E|1 year ago
_tk_|1 year ago
betaby|1 year ago
bgrainger|1 year ago
wwilim|1 year ago
[deleted]
olig15|1 year ago
pjc50|1 year ago
verve_rat|1 year ago