I was watching a livestream recently, and the person turned to grab something out of frame, and for a split second when their face was near the border, the filter glitched out and gave flashes of the person's real face. I was truly shocked as I had been watching for about five minutes with zero inclination that the person was essentially wearing a digital mask.
My partner was showing me a Doja Cat TikTok last night where she was singing an old song with an “old age” face filter on, and the effect was stunning. Super realistic wrinkles, contours and skin texture, it never glitched out and had no visible edges. We are truly through the looking glass.
> overall average accuracy was near chance at 48 percent, although individuals’ accuracy varied considerably, from a low of around 25 percent to a high of around 80 percent.
People with a 25% score are basically as good as recognizing fakes as people with an 80% score. The former group just prefer the fakes to the real..
The community of people making deepfakes even approaching this quality is astonishingly small (think 3-4 people), requires absurd hardware (either A100s or A6000s), and the Luke one is even more of a outlier as this deepfaker has the support of ILM, one of the best VFX studios in the industry. All this is to say I'm not especially scared — especially when these "best of the best" deepfakes still have obvious tells.
Not weighing in on whether one should be frightened or not, but just pointing out that what you said would have applied to the computer graphics industry about 30 years ago. The interesting thing about computers is they tend to get faster over time.
If Disney can't make Luke Skywalker, one of their most iconic characters, look good in a show with a budget the size of the Mandalorian, I can't say I'm too worried for this technology yet.
Everyday misinformation techniques are far more pernicious - out-of-context sound bites, selective coverage, framing in an uncharitable way, interviewing non-experts, etc etc.
This is just an iteration of the same overreaction that has been occurring since the days of splicing audio tape together.
If anything, I would argue malicious editing of REAL footage, which already occurs in media outlets today, is far more frightening than the possibility of having a deepfake video, though it might be another tool in that slimy toolbox.
Convincing enough deep fake + the 24/7 news industry + Twitter rage machine combined will at some point overreact way too fast and too hard and someone will get actually hurt, maybe even killed.
Actors have had clauses for not using their likeness for a long time now, if their face is 3D scanned, the use cases are carefully limited. They can't just put Brad Pitt in a random commercial even though they have the scans of his face and body and can find a voice actor.
There's a subplot in The Newsroom where a producer edits an interviewee out of context, leading to the network broadcasting false accusations of serious war crimes. The producer wasn't careful enough, and a clock visible in the background is seen to jump around. Without that clock it would have been undetectable.
Even malicious editing has been around a long time, but the ease and accessibility, coupled with social media proliferation, is on another level. I'm not sure to what extent this matters yet when mere text on a screen can fool certain demographics. Is doctored footage/images a moot point if the target viewers are going to believe it anyway?
What we need is better inoculation to psycho misinformation politics
I take a similar approach, but from a different angle: the widespread awareness of deepfakes is useful because people will stop relying on video "evidence" as evidence.
We already do this with photos. We don't necessarily worry so much about what's in the photo as who claims the photo is real and what the chain of custody was.
The same will be true for video evidence, even if or especially if we start stamping videos with tamper-proof signatures.
And then there's doubles: when you just find somebody who looks exactly like the real person, and get them to do some acting.
A deepfaked video or photoshop can be debunked because we're all aware that the technology exists to fake it; all we have to do is show evidence that it's not true. And at the same time, there will always be people who simply want to believe something and so will refuse any evidence to the contrary. You don't need a Deepfake for someone to believe that Hillary Clinton is running a gang of pedophiles out the nonexistent basement of a pizza parlor in the DC suburbs.
I've never been worried about this because I just assume that official outlets and websites will use some kind of verifiable encryption keys. Makes official official and everything else suspect.
Is this too simplistic an outlook? Are there different options? I'm not sure but I know that third party unverified content is always suspect. First party content less suspect. Verified content is ... verified. Still could be deepfake but then the risk is on a loss of trust.
It's worth noting that society existed for thousands of years without photography, audio recording, or videography. As you allude to, society is held together via trust networks, and these are independent of the prevailing media.
There are huge opportunities here for hashing and watermarking that guarantees videos are unaltered from the original recording taken by a physical device. An iphone already has all the hardware it needs to ensure it cannot be fooled by a flat image and to calculate and insert a cryptographic hash while recording.
We will lose the battle against deepfakes, but we can create a subcategory of video proven as non-fake and win the war.
They’ve been trying to scare us with deepfakes for so long now. It’s such a yawn. They create such a narrow image type (face on blurry background) and say “look how real they look!”
Same with deepfake videos, which could always be faked with good FX and one of the reasons why videos with no chain of custody don’t become evidence in court already!
Am I alone in having thought that the term ‘deepfake’ only applied to NN/deep learning-generated imagery, rather than just plain CG, as implied in this article?
One one hand - I would say that everything that can be abused will be abused. On the other hand I would say, people (after a time of adaption) will learn to not trust any moving image anymore.
Or any still image for that. But on the other hand - we are already theoretically and practically able to generate very convincing fake images, if we would want to. We could generate fake stills from a surveillance video (or a smartphone video) of a fake politician doing cocaine. Or interacting in a compromising way with a prostitute. Why do we not see much more of things like that?
Probably because it just might be more difficult to create a believable story around such a fake that does not break down when looking at the story more than 10 seconds.
So maybe the problem will not be as big? I don't know. I have no idea how easy it will become to create truly believable fakes.
Not sure how people will adapt and if trust in anything might break down in the process. But what I do not believe is that tech industry creating rules for itself will be a deterrent against abuse of any technology.
How many people are convicted on 480p or less janky security footage? Making believable bad fakes is trivial when high resolution fakes are only moderately difficult.
If it hasn't happened yet, it will soon - people will be submitting modified doorbell camera and home security footage to police. Or someone's compromised home network will record a deep fake overlay of otherwise legitimate footage, and the owner won't be complicit. In a situation with, say, a $5,000 or more item, or a schedule 1 or 2 substance stolen, a person could be subjected to damning evidence of them committing a serious felony, for potentially less difficulty and risk than swatting.
I see many people here worried about the disinformation potential of deepfakes. I think that that worry fundamentally ignores the fact that we'll just stop considering videos evidence.
A century ago, you could trust photos. Then, Photoshop came out, and the world didn't end, we just learned that we can no longer trust photos. Now we just won't trust anything, and it'll be one party's word against the other's.
Sure, this isn't as good as living in a world with objective proof, but I think we'll grow out of the short-term issues and find different ways of verifying truth.
If Hollywood's use of deepfake Luke Skywalkers gives "more truth-oriented" folks a good-enough awareness of the technology, and fact-free stuff like the Pizzagate conspiracy is good enough for the "less truth-oriented" folks...then deepfakes aren't much of a threat.
Once this is more accessible for smaller studios, with smaller budgets, then actors will be obsolete.
10y from now, we will be able to simulate any human action or sound or gesture on screen with commodity hardware. I can have a 75m USD cast for my film, for the price of a couple GPUs spun for a few days. Crazy.
You still need people who can act. Someone has to make the characters act, and you don't want it to suck, so somebody needs to be skilled at acting, even if they don't end up doing the physical acting.
But more to the point, there is no need for constant deepfakes. You can just get regular people to act in your movie. Some movies might need deepfakes for the same reason that animes are really popular. But there's plenty of movies that would not benefit from deepfakes at all.
Fully animated actors can play just as effective and convincing as flesh-and-blood actors, as shown by Disney or Pixar. But is also requires immense amounts ow works and talent - neither which is cheap.
Star Wars certainly didn't use deepfakes because they though it was cheaper or easier.
It really won't. That would just be a cartoon with a realistic art style. You still need almost every other part of a film. You still need an original performance to deep fake. This isn't an AI actor or a simulation, it's a post effect.
I'm not sure what failure to reliably discriminate between real pictures of people I don't know and therefore wouldn't be able to recognize and (what are basically composite) computer-generated people who don't exist is supposed to prove. Artists who can paint a picture of someone who doesn't exist that I can't tell from pictures of people who do exist, and I could probably piece together somebody who doesn't exist in photoshop from pieces of people who do.
Also, a weird-ass set of conclusions here:
> People are also excellent at incorporating context and background knowledge into their judgments in ways that computers mostly aren’t, and in ways that the sorts of tasks in these studies mostly don’t assess. One exception is a pair of deepfake videos of Vladimir Putin and Kim Jong-un included in the video study. People were much better than the model at classifying these videos. Whereas a majority of participants correctly spotted these fakes, the model was way off, highly confident that they were real.
> Here, people may have taken into account factors like their memories of previous statements made by these leaders, or knowledge of what their voices sound like, or the likelihood that they would say the sorts of things shown in the video: all things not included in the model at all.
Couldn't it just be because they were bad imitations of people they recognized? Isn't the entire ominous point of "deepfakes" to deceive people by presenting them with people the recognize doing things they didn't do? Looks like people still don't have a problem recognizing those as false.
The only good "deepfakes" I've seen are ones that have been extensively touched up manually, using the automatic part as raw material. New techniques have certainly made that a lot cheaper.
This may be an unpopular opinion, but I think deepfakes could be a very good thing for society. Imagine if we got to the point where it was completely impossible to distinguish fake images and videos from real ones. It would be a huge blow against surveillance capitalism and the surveillance state.
If every video out there is suddenly a potential forgery, people will grow to become more skeptical of things they see unless they see those things in person in the meatspace.
It's almost as if we're going to go back to the time before cameras and audio recordings. What makes things like deepfakes so scary right now is that people overwhelming trust images and videos by default. Once we shift from automatically trusting images and videos to automatically distrusting them, things like revenge porn and viral videos of people saying or doing embarrassing things will lose most of their power.
[+] [-] ceroxylon|4 years ago|reply
[+] [-] Uehreka|4 years ago|reply
[+] [-] andrewgioia|4 years ago|reply
Here's his video comparison: https://www.youtube.com/watch?v=wrHXA2cSpNU
[1] https://www.engadget.com/lucasfilm-hires-shamook-054904077.h...
[+] [-] victor9000|4 years ago|reply
[+] [-] thomasahle|4 years ago|reply
People with a 25% score are basically as good as recognizing fakes as people with an 80% score. The former group just prefer the fakes to the real..
[+] [-] scarmig|4 years ago|reply
[+] [-] lupire|4 years ago|reply
[+] [-] vincentriemer|4 years ago|reply
[+] [-] gdubs|4 years ago|reply
[+] [-] pradn|4 years ago|reply
Everyday misinformation techniques are far more pernicious - out-of-context sound bites, selective coverage, framing in an uncharitable way, interviewing non-experts, etc etc.
[+] [-] lupire|4 years ago|reply
A6000 is only $50/day rental.
[+] [-] zehaeva|4 years ago|reply
I'm thinking specifically of 2002's S1M0NE[0].
I can't wait for this future.
[0]https://en.wikipedia.org/wiki/Simone_(2002_film)
[+] [-] echelon_musk|4 years ago|reply
https://m.imdb.com/title/tt1821641/
[+] [-] sillyquiet|4 years ago|reply
This is just an iteration of the same overreaction that has been occurring since the days of splicing audio tape together.
If anything, I would argue malicious editing of REAL footage, which already occurs in media outlets today, is far more frightening than the possibility of having a deepfake video, though it might be another tool in that slimy toolbox.
[+] [-] theshrike79|4 years ago|reply
Convincing enough deep fake + the 24/7 news industry + Twitter rage machine combined will at some point overreact way too fast and too hard and someone will get actually hurt, maybe even killed.
Actors have had clauses for not using their likeness for a long time now, if their face is 3D scanned, the use cases are carefully limited. They can't just put Brad Pitt in a random commercial even though they have the scans of his face and body and can find a voice actor.
[+] [-] space_ghost|4 years ago|reply
[+] [-] slothtrop|4 years ago|reply
What we need is better inoculation to psycho misinformation politics
[+] [-] unsupp0rted|4 years ago|reply
We already do this with photos. We don't necessarily worry so much about what's in the photo as who claims the photo is real and what the chain of custody was.
The same will be true for video evidence, even if or especially if we start stamping videos with tamper-proof signatures.
[+] [-] 0xbadcafebee|4 years ago|reply
And then there's doubles: when you just find somebody who looks exactly like the real person, and get them to do some acting.
A deepfaked video or photoshop can be debunked because we're all aware that the technology exists to fake it; all we have to do is show evidence that it's not true. And at the same time, there will always be people who simply want to believe something and so will refuse any evidence to the contrary. You don't need a Deepfake for someone to believe that Hillary Clinton is running a gang of pedophiles out the nonexistent basement of a pizza parlor in the DC suburbs.
[+] [-] sgfgross|4 years ago|reply
[+] [-] sosuke|4 years ago|reply
Is this too simplistic an outlook? Are there different options? I'm not sure but I know that third party unverified content is always suspect. First party content less suspect. Verified content is ... verified. Still could be deepfake but then the risk is on a loss of trust.
[+] [-] vannevar|4 years ago|reply
[+] [-] Joeri|4 years ago|reply
We will lose the battle against deepfakes, but we can create a subcategory of video proven as non-fake and win the war.
[+] [-] ed25519FUUU|4 years ago|reply
Same with deepfake videos, which could always be faked with good FX and one of the reasons why videos with no chain of custody don’t become evidence in court already!
[+] [-] monadmoproblems|4 years ago|reply
[+] [-] doctorhandshake|4 years ago|reply
[+] [-] dandellion|4 years ago|reply
[+] [-] kevincox|4 years ago|reply
[+] [-] sdoering|4 years ago|reply
Or any still image for that. But on the other hand - we are already theoretically and practically able to generate very convincing fake images, if we would want to. We could generate fake stills from a surveillance video (or a smartphone video) of a fake politician doing cocaine. Or interacting in a compromising way with a prostitute. Why do we not see much more of things like that?
Probably because it just might be more difficult to create a believable story around such a fake that does not break down when looking at the story more than 10 seconds.
So maybe the problem will not be as big? I don't know. I have no idea how easy it will become to create truly believable fakes.
Not sure how people will adapt and if trust in anything might break down in the process. But what I do not believe is that tech industry creating rules for itself will be a deterrent against abuse of any technology.
[+] [-] robbedpeter|4 years ago|reply
If it hasn't happened yet, it will soon - people will be submitting modified doorbell camera and home security footage to police. Or someone's compromised home network will record a deep fake overlay of otherwise legitimate footage, and the owner won't be complicit. In a situation with, say, a $5,000 or more item, or a schedule 1 or 2 substance stolen, a person could be subjected to damning evidence of them committing a serious felony, for potentially less difficulty and risk than swatting.
[+] [-] stavros|4 years ago|reply
A century ago, you could trust photos. Then, Photoshop came out, and the world didn't end, we just learned that we can no longer trust photos. Now we just won't trust anything, and it'll be one party's word against the other's.
Sure, this isn't as good as living in a world with objective proof, but I think we'll grow out of the short-term issues and find different ways of verifying truth.
[+] [-] bell-cot|4 years ago|reply
[+] [-] adenozine|4 years ago|reply
10y from now, we will be able to simulate any human action or sound or gesture on screen with commodity hardware. I can have a 75m USD cast for my film, for the price of a couple GPUs spun for a few days. Crazy.
[+] [-] onion2k|4 years ago|reply
Only if you want the people you've copied to sue you for breaching their personality rights (image rights in other countries). https://en.wikipedia.org/wiki/Personality_rights#United_Stat...
[+] [-] 0xbadcafebee|4 years ago|reply
But more to the point, there is no need for constant deepfakes. You can just get regular people to act in your movie. Some movies might need deepfakes for the same reason that animes are really popular. But there's plenty of movies that would not benefit from deepfakes at all.
[+] [-] goto11|4 years ago|reply
Star Wars certainly didn't use deepfakes because they though it was cheaper or easier.
[+] [-] lancesells|4 years ago|reply
[+] [-] jayd16|4 years ago|reply
[+] [-] fuckcensorship|4 years ago|reply
[+] [-] pessimizer|4 years ago|reply
Also, a weird-ass set of conclusions here:
> People are also excellent at incorporating context and background knowledge into their judgments in ways that computers mostly aren’t, and in ways that the sorts of tasks in these studies mostly don’t assess. One exception is a pair of deepfake videos of Vladimir Putin and Kim Jong-un included in the video study. People were much better than the model at classifying these videos. Whereas a majority of participants correctly spotted these fakes, the model was way off, highly confident that they were real.
> Here, people may have taken into account factors like their memories of previous statements made by these leaders, or knowledge of what their voices sound like, or the likelihood that they would say the sorts of things shown in the video: all things not included in the model at all.
Couldn't it just be because they were bad imitations of people they recognized? Isn't the entire ominous point of "deepfakes" to deceive people by presenting them with people the recognize doing things they didn't do? Looks like people still don't have a problem recognizing those as false.
The only good "deepfakes" I've seen are ones that have been extensively touched up manually, using the automatic part as raw material. New techniques have certainly made that a lot cheaper.
[+] [-] tharne|4 years ago|reply
If every video out there is suddenly a potential forgery, people will grow to become more skeptical of things they see unless they see those things in person in the meatspace.
It's almost as if we're going to go back to the time before cameras and audio recordings. What makes things like deepfakes so scary right now is that people overwhelming trust images and videos by default. Once we shift from automatically trusting images and videos to automatically distrusting them, things like revenge porn and viral videos of people saying or doing embarrassing things will lose most of their power.