Lucasfilm's subsidiary Industrial Lights and Magic is known for leading visual effects on actors's appearances to help make films that cannot be realized without such technology. For instance, Martin Scorsese entrusted them to "youthen" the leading actors in "The Irishman" so that Robert Deniro et al can play their characters at a younger age without wearing a red-ball "clown-nose" tracker. "Star Wars: Rouge One" practically reanimates Peter Cushing to continue and expand the involvement of Grand Moff Tarkin in the Star Wars saga. These processes are painstaking, and artists sweat over details on a frame-by-frame basis to negate the "uncanny valley" of artificial human likenesses.
Obviously, the labor-intensive nature of today's CGI techniques drive up production costs. Meanwhile, the deepfakes on YouTube provide a convincing enough rendition of likenesses without actual actors, all produced on consumer-level GPUs. This presents a huge potential to save costs and the benefits are clearly enticing to film productions.
As Hollywood gravitates towards blockbuster franchises, productions will want to bring the same ensemble of actors (or at least their likenesses) as long as possible. While moviegoers may be unsettled by seeing "reanimated" dead actors like in Rouge One, they still may hope to see franchise actors to look consistently youthful or attractive on screen. Deepfakes may be more relied upon to provide that effect.
My question is, how would the cost structure works. Can I now hire an actor that looks 90% like someone I have in mind, and then Deepfake it to look 99.9%, and save on actor's cost?
It works when you are trying to do that on actor that are no longer with us, but what about actors that are still alive?
It is so promising that this technology is reaching the masses and letting random youtubers compete with the best. Hopefully this type of tech further decentralizes the content creation from Hollywood to other parts of the world.
Give it another decade or so, and I can see Lucasfilm/Disney making full feature films starring only deepfake 'clones' of the original trilogy characters in their younger/O.T. forms.
The worst thing a lot of these deepfake folks do with movie clips is hire impersonators to try to make it more realistic. The problem is that a) the impersonator is usually off a bit in timing or character and b) the soundstage is nothing like the rest of the movie, it just sounds like cuts to a podcast. The result just doesn't work.
While even the new ones are not perfect, they are way better than what's in the movies/shows. Good they decided to hire the artist instead of just beating him with lawyers.
Lucus has always been good with things like this. Star Wars games pre-disney were generally easy to mod, and fan fiction writers didn't have to be too careful about getting sued.
As long as the creator wasn't making money from it.
It's one of the reasons why the franchise survived for so many years without new movies.
To the people complaining that what you see is no longer believable: there is a way out: signatures. It is time to start pressuring public people to digitally sign whatever they say.
I'm not really into the tech behind Deep Fake, but doesn't the whole credit go to the tool? Or is it that one needs to adjust the tool somehow to produce decent fakes?
"Shamook is the one who “fixed” Luke Skywalker’s cameo in The Mandalorian to the tune of 1.9 million videos..."
I understand this is a mistake, we all make them, but where are the editors at The Verge? If a reader can find this with a cursory look-over, shouldn't they find it also? I couldn't have handed this in as a high school essay, so I'd imagine it wouldn't get past an editor at a large magazine. Maybe it's just me but at least from what I see personally there's loads of errors small and larger in the news these days. It's weird.
I don't understand. What's the original vs deepfake comparison about? I know nothing of Star Wars (sorry), is there a third video it's based on or something?
Well, why are good actors so sought-after? What makes someone an actor whose likeness people seek and what makes a B-tier actor?
Part of it is inertia and random celebrity status, having an attractive or interesting face etc., but part of it is also the raw knowledge of when to apply certain microexpressions, how to gesture etc. i.e. how to do the acting itself. To be a convincing, charismatic etc. actor it's not enough to wear a digital mask of a celebrity, the underlying actor still needs to act well. That may not be so important for certain types of shallow movies, but it certainly is for deeper drama films etc.
It's similar to today's text generation where you may be able to generate sports game reports, user's manuals or travel brochures etc. but not really those where you need high level decisions, like applying the appropriate expressions to a real-world event, taking into account all the context, like writing a poem about your feelings reflecting on some recent real-world event.
I'm not saying humans have a magical power that can't be implemented in silicon.
What I'm saying is that deepfakes as they are today are not sufficient to replace actors. You'd need a higher level puppeteering AI that would take the whole storyline and script into account to come up with the right ways to express the appropriate emotions at that moment in the film and could take the director's instructions regarding his vision of how the drama should unfold etc.
There is a company doing this for video game voices. The voice actors who provide training data get royalties. I hope that becomes a standard for deep fakes and the ML in general, as opposed to how copilot (and the rest of the industry) generally just takes whatever they get their hands on as free train training data.
I doubt Hollywood would want this, but I'd love to be entertained by choosing some base story, then be able to pick the lead actors and perhaps setting and mood. Deep fakes get a ways toward this. This is basically what remakes are.
Is it just me who thinks the title is sensationalised? I can barely see any difference between the two, and the one made by the YouTube builds on top of the original one.
It's not just you. I can't tell the difference either.
Maybe it's just something someone has to explain to us what to look for. It could be a curse to know why some people think the deepfake is better. I once talked to a graphics artist about why they thought some effect looked bad because I didn't see it. They explained it in detail. Now I can't unsee what they were talking about and can easily spot that mistake.
I don't know. Maybe it's better not knowing and enjoying the results or knowing and each time you see it thinking: "Hey, they made that mistake"
I think the key difference is how the eyes are “reanimated” in the deepfakes. Rendering realistic eyes (with that trademark “spark of life”) has always been a challenge in real life emulating CGI, but now these new clips kind of capture the spark in the actors eyes much better. Luke Skywalker's eyes in the original clip kind of look dead plastic, the kind you’d find in real-time gameplay (not to mention this nose also looks like the product of bad rhinoplasty). The deepfake does this much better, in my opinion.
Elsewhere in this thread, someone posted a similar comparison clip about The Irishman and the difference is even more pronounced there. De Niro’s eyes actually look the age they’re supposed to be in that scene. The original scene had two key problems, in my option; one, they decided not to touch the eye and focused on smoothening the skin (I assume this is because of the technical limitations of doing the deaging in painstaking CGI); two, and this is a little more of a mystery, is that they thought they could get away with using De Niro’s current 80 year old voice on a character that is supposed to be 35 (ages are ballpark numbers). The raspy voice of an old man is just not something you expect out of a supposedly much younger and healthier man. They should have just gotten a voice actor who could do a convincing impression of De Niro in the 70s and dubbbed him in.
Building "on top of the original one" is not an asset in the deepfake process, it's a hindrance, because then the deepfake inherits the unnatural 3D facial animation, which is most of the reason why traditional VFX 3D facial replacement works so poorly.
The deep fake one looks very weird to me. The mouth is sometimes not closed and there is minimal movement on the upper lip on words that would need more.
The deepfakes look much better than the original, especially Leia, whose CGI recreation in Rogue One looked odd from the beginning. Tarkin is ok just like Luke, especially their eyes which now seem to be looking to whom they're talking to, thanks to better reflections. Luke's mouth however is still unrealistic when matching the speech; for example at 0:32 when Luke says "He wants your permission", the lips don't even touch to create the "P" sound.
I've seen some deep fakes that are very convincing, especially if you show them to someone not knowing about them. One example would be this one in which Sean Connery's face was replaced by Burt Reynold's in James Bond:
https://www.youtube.com/watch?v=foqeQM-7PSg
Truly amazing what can be done with consumer grade stuff nowadays.
ILM isn't dumb - this is a talenthire for sure. Amazing work from this guy and dedication to the niche (and they likely have some ideas kicking around in Marvel/Lucas writing rooms that are about bringing alllll the olds back to life)
I know the narrative is to be scared of this tech (maybe even push towards legislation of it). But me personally I just find these things amazingly awesome and super cool.
This reminds me of Bojack Horseman plot-line where they scanned every actor on set in case they die so they can recreate them in virtual for the movie.
[+] [-] coolandsmartrr|4 years ago|reply
Obviously, the labor-intensive nature of today's CGI techniques drive up production costs. Meanwhile, the deepfakes on YouTube provide a convincing enough rendition of likenesses without actual actors, all produced on consumer-level GPUs. This presents a huge potential to save costs and the benefits are clearly enticing to film productions.
As Hollywood gravitates towards blockbuster franchises, productions will want to bring the same ensemble of actors (or at least their likenesses) as long as possible. While moviegoers may be unsettled by seeing "reanimated" dead actors like in Rouge One, they still may hope to see franchise actors to look consistently youthful or attractive on screen. Deepfakes may be more relied upon to provide that effect.
[+] [-] shadowtree|4 years ago|reply
https://www.youtube.com/watch?v=dHSTWepkp_M
Just look at it, the CGI DeNiro looks like from the Polar Express.
[+] [-] ksec|4 years ago|reply
It works when you are trying to do that on actor that are no longer with us, but what about actors that are still alive?
[+] [-] mensetmanusman|4 years ago|reply
It reminds me of the ‘what’s in the box’ 2009 short made with available cgi assets https://m.youtube.com/watch?v=IU_reTt7Hj4
[+] [-] lmilcin|4 years ago|reply
This is going to be one more dimension to misinformation on the net.
Now you will see videos of politicians saying something and even then you cannot be sure whether this is actual video or a fake.
[+] [-] jordanab|4 years ago|reply
[+] [-] jcims|4 years ago|reply
https://youtu.be/A8TmqvTVQFQ?t=52
That said, one way it *does* work is with novel scenes filmed with a good impersonator, the outcome can be pretty remarkable:
https://www.tiktok.com/@deeptomcruise/video/6957456115315657...
https://www.youtube.com/watch?v=krAU3C9jhj8
https://www.youtube.com/watch?v=ybasoc6LxIU
https://www.youtube.com/watch?v=VWrhRBb-1Ig
[+] [-] coldcode|4 years ago|reply
[+] [-] andyp-kw|4 years ago|reply
As long as the creator wasn't making money from it.
It's one of the reasons why the franchise survived for so many years without new movies.
[+] [-] marcodiego|4 years ago|reply
[+] [-] sdevonoes|4 years ago|reply
[+] [-] mdrzn|4 years ago|reply
[+] [-] easterncalculus|4 years ago|reply
I understand this is a mistake, we all make them, but where are the editors at The Verge? If a reader can find this with a cursory look-over, shouldn't they find it also? I couldn't have handed this in as a high school essay, so I'd imagine it wouldn't get past an editor at a large magazine. Maybe it's just me but at least from what I see personally there's loads of errors small and larger in the news these days. It's weird.
[+] [-] matsemann|4 years ago|reply
[+] [-] wodenokoto|4 years ago|reply
There’s no ground truth to compare with other than the Luke skywalker character in the original Star Wars movies and which one looks most realistic
[+] [-] nickthegreek|4 years ago|reply
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] mindvirus|4 years ago|reply
Do we have different actors deepfaked in for different markets?
Do actors even act anymore? Do companies just pay actors for their likeness and do the rest?
Do we get to a point where you can choose who is acting in a film you're watching?
How are deepfaked voices? Can we substitute audio as well?
I understand that deepfakes are still a bit of a manual process, but presumably that will change.
[+] [-] bonoboTP|4 years ago|reply
Part of it is inertia and random celebrity status, having an attractive or interesting face etc., but part of it is also the raw knowledge of when to apply certain microexpressions, how to gesture etc. i.e. how to do the acting itself. To be a convincing, charismatic etc. actor it's not enough to wear a digital mask of a celebrity, the underlying actor still needs to act well. That may not be so important for certain types of shallow movies, but it certainly is for deeper drama films etc.
It's similar to today's text generation where you may be able to generate sports game reports, user's manuals or travel brochures etc. but not really those where you need high level decisions, like applying the appropriate expressions to a real-world event, taking into account all the context, like writing a poem about your feelings reflecting on some recent real-world event.
I'm not saying humans have a magical power that can't be implemented in silicon.
What I'm saying is that deepfakes as they are today are not sufficient to replace actors. You'd need a higher level puppeteering AI that would take the whole storyline and script into account to come up with the right ways to express the appropriate emotions at that moment in the film and could take the director's instructions regarding his vision of how the drama should unfold etc.
[+] [-] vollmond|4 years ago|reply
I assume that would eventually get to generating a totally new person, rather than modeling after a specific actor who costs money.
[+] [-] 6gvONxR4sf7o|4 years ago|reply
[+] [-] gedy|4 years ago|reply
[+] [-] mmkos|4 years ago|reply
[+] [-] dang|4 years ago|reply
[+] [-] AnIdiotOnTheNet|4 years ago|reply
[+] [-] nocturnial|4 years ago|reply
Maybe it's just something someone has to explain to us what to look for. It could be a curse to know why some people think the deepfake is better. I once talked to a graphics artist about why they thought some effect looked bad because I didn't see it. They explained it in detail. Now I can't unsee what they were talking about and can easily spot that mistake.
I don't know. Maybe it's better not knowing and enjoying the results or knowing and each time you see it thinking: "Hey, they made that mistake"
[+] [-] MajorBee|4 years ago|reply
Elsewhere in this thread, someone posted a similar comparison clip about The Irishman and the difference is even more pronounced there. De Niro’s eyes actually look the age they’re supposed to be in that scene. The original scene had two key problems, in my option; one, they decided not to touch the eye and focused on smoothening the skin (I assume this is because of the technical limitations of doing the deaging in painstaking CGI); two, and this is a little more of a mystery, is that they thought they could get away with using De Niro’s current 80 year old voice on a character that is supposed to be 35 (ages are ballpark numbers). The raspy voice of an old man is just not something you expect out of a supposedly much younger and healthier man. They should have just gotten a voice actor who could do a convincing impression of De Niro in the 70s and dubbbed him in.
[+] [-] yourenotsmart|4 years ago|reply
[+] [-] knuthsat|4 years ago|reply
[+] [-] aazaa|4 years ago|reply
[+] [-] bluebubble56|4 years ago|reply
[deleted]
[+] [-] squarefoot|4 years ago|reply
[+] [-] remir|4 years ago|reply
Truly amazing what can be done with consumer grade stuff nowadays.
[+] [-] diegoperini|4 years ago|reply
[+] [-] ChrisArchitect|4 years ago|reply
[+] [-] bick_nyers|4 years ago|reply
https://en.m.wikipedia.org/wiki/Simulacra_and_Simulation
[+] [-] croes|4 years ago|reply
[+] [-] tvirosi|4 years ago|reply
[+] [-] me_me_me|4 years ago|reply