It was so jarring I couldn't watch it and switched to Japanese with subtitles. Their mouths looked so stiff and unnatural and out of sync with their expressions that all me and my wife could talk about was how bad it looked.
You can see a lot of open mouth/tongue stuff being skipped. Dealing with the tongue and the inside of the mouth is a huge problem with this sort of visual dubbing. Using traditional techniques, you can model teeth and gums as rigid bodies, and faces as rubber sheets (to first approximation), but tongues, for which you typically have no visual reference in any given shot, are much more difficult to model, and continuously, subtly, on the move. "AI" is the general answer to this problem nowadays, but even ML-based systems struggle to deal with the tongue issue while trying to reconcile visual appearance with animation fidelity.
> Their mouths looked so stiff and unnatural and out of sync with their expressions that all me and my wife could talk about was how bad it looked.
Everything old is new again. Classic Italian films were almost always dubbed by their original actors, to the point that Fellini sometimes didn't even write portions of the dialog till after filming was done. The actors just said random stuff during filming. Originally this was mainly a technology limitation (sets were too noisy), then it became a cost-savings measure and cultural tradition to how films were made in Italy.
I always find it a bit of a distraction when I haven't watched an Italian film in a while, but eventually get used to it.
It was quite jarring in Amazon's Citadel Diana. The voices were ever so slightly out of sync with the lip movements, and the audio sounded like the studio recording hadn't been processed to match the environment.
On the other hand, as a lover of film, if this will bring more audience to foreign films, then let's give it a try.
This is very much in the same vein as colorizing black and white movies, pan-and-scan for video releases (and now zoom for 4:3 content on widescreen), and dubbing instead of subtitles. Every one of those things brought in more viewers of the content, exposing more people to those films.
I want to be on board with the “bring more audience” but the truth is knowing how this industry works, any success with AI on this front will lead almost immediately to a total drying up of funding for professionals engaging in this work currently. I hope I’m wrong, I really do.
I grew up watching foreign movies almost exclusively dubbed. French TV in the 1980s-2000s dubbed everything; it was nearly impossible to watch stuff with the original audio track. And I will say something that will probably surprise most of you: up until age 10-12 I never noticed the lips didn't match precisely the voices and I never even realized it was dubbed(!)
Even when I learned most of what I was watching was dubbed (there is so much American content on TV) it did not seem obvious to me. It certainly never bothered me. Maybe my brain subconsciously realized watching the lips was not giving out much information, so I learned to ignore lips.
Even today as an adult, if I don't know the actors and I watch a French dub, I am not able to instantly tell it is a dub. I have to watch carefully for at least 10-20 seconds to make a guess. Maybe this speaks more of the fact that French dubs are actually really well made (they will choose translations so that French word cadence matches the English word cadence). On the other hand I have watched other dubs, especially non-English movies dubbed in English, where it's blatant that they were dubbed and it is very distracting.
In any case, I always get a kick when I tell my American friends that I watched US content dubbed all my childhood and it never bothered me :-)
German dubs are smilarly well done, but the result often sounds slightly like the original language in tone and cadence, which is recognizable if you know what to look for. Someone I know can do a quite funny exaggerated version of German-from-English.
Extreme example: children say "maMA" (strongly stressed on the last syllable, opposite of German) in dubbed French movies. It's actually a little annoying, haha.
I would love a way to regularly discover and watch some top, 'hit' TV shows or movies from outside of America or Australia (my home country), or sometimes England. I have no issues watching things with subtitles, but it is quite difficult to organically find things to watch, that perhaps have 'not English' as their primary language.
For example, I will now look up the UFO Sweden show to see if I might like to watch it -- because I discovered it through here.
I have tried subscribing to a few different regular streaming services, but none seemed to work even if I pointed them in the right direction. I'm really not particular about which country of origin the production has, so long as it's "the best" or "very popular" in recent times from that place, it's worth me checking the genre and style to see if it piques my interest.
Any advice on this? [ nb. I have a similar issue with podcasts! ]
You might enjoy https://easterneuropeanmovies.com/, https://sovietmoviesonline.com/ or https://asian-movies-online.com/. They have both movies that are popular hits and that have been important to the craft and history of the art. Subtitles are usually from open sub-style sources so not professional quality, but I find it sometimes help with learning the language when it's more literally translated than idiomatically.
I'm learning a language, and I'd love to be able to filter Netflix titles by language, but they absolutely refuse to do it (I assume because allowing users to filter content would expose how bare the offerings are).
As someone who doesn't speak Swedish, what makes this particular passage interesting, and what is it about the Norrköping dialect makes it interesting?
(I have attempted to study Swedish briefly, but I feel like I need to do more sustain discovery of content to consume in order to keep me engaged long term)
I support this simply just on the fact that maybe they wouldn't have remade Danish film "Speak no evil" (2022) which was such a great movie into one of the worst movies I've seen this year, the American film "Speak no evil" (2024)
The very few shots they showed in the demo video, which one could assume were the best ones they wanted to highlight, do not look good, like it's not as bad as Ice Ice Matrix [1] but it does not look good.
They're admittedly pretty cagey with the parts they're willing to show, but the parts they did show look fine to me. Are you referring to the parts when they showed the 3d mask overlay on top of the characters?
Man, this is so much better than many Bollywood movies that are re-dubbed. When the actors go back to the sound studio to re-record their lines, they don't match up with their lips exactly, and it can be off-putting.
A bit off-topic maybe: I really think the earlier film made by the same production team (Crazy Pictures), "The Unthinkable" [1], is a much better film than "Watch the Skies".
But apparently IMDb does not agree:
"The Unthinkable", IMDb rating 6.0
"Watch the Skies", IMDb rating 6.4
Did they literally ask a machine "visually dub subtitles into this movie file for me please" and get a result? No? They didn't use AI then. They used a purpose built model in a purpose built tool to perform a single algorithmic function. how that algorithm works or what technologies it's similar to are not relevant.
We've had things we call "AI" since the 1950s. We only had text prompting as an interaction mode since ChatGPT came out in 2022. Why is prompting your test for whether something is "AI" or not?
As someone that grew up in a Latin America country watching dubbed movies my whole life I welcome this technology. Dubbed movies are usually terrible with subpar performances and recycled voices.
It sucks it will kill a large field for actors, but AI has the capacity to be much better with voices and performances that mimic the actors.
> Notably, the original actors recorded their own dialogues in English in a sound booth — Flawless AI's technology merely altered the movements of their lips in the movie.
I support this, if only because I prefer dubs over subs, and this will make dubs much more worthwhile for the studio to get right, as well as better viewer experience.
[+] [-] sleepydog|11 months ago|reply
It was so jarring I couldn't watch it and switched to Japanese with subtitles. Their mouths looked so stiff and unnatural and out of sync with their expressions that all me and my wife could talk about was how bad it looked.
[+] [-] lambdaone|11 months ago|reply
[+] [-] SirFatty|11 months ago|reply
[+] [-] js2|11 months ago|reply
Everything old is new again. Classic Italian films were almost always dubbed by their original actors, to the point that Fellini sometimes didn't even write portions of the dialog till after filming was done. The actors just said random stuff during filming. Originally this was mainly a technology limitation (sets were too noisy), then it became a cost-savings measure and cultural tradition to how films were made in Italy.
I always find it a bit of a distraction when I haven't watched an Italian film in a while, but eventually get used to it.
[+] [-] chedabob|11 months ago|reply
[+] [-] unknown|11 months ago|reply
[deleted]
[+] [-] aaron695|11 months ago|reply
[deleted]
[+] [-] jedberg|11 months ago|reply
On the other hand, as a lover of film, if this will bring more audience to foreign films, then let's give it a try.
This is very much in the same vein as colorizing black and white movies, pan-and-scan for video releases (and now zoom for 4:3 content on widescreen), and dubbing instead of subtitles. Every one of those things brought in more viewers of the content, exposing more people to those films.
[+] [-] voltaireodactyl|11 months ago|reply
[+] [-] mrb|11 months ago|reply
Even when I learned most of what I was watching was dubbed (there is so much American content on TV) it did not seem obvious to me. It certainly never bothered me. Maybe my brain subconsciously realized watching the lips was not giving out much information, so I learned to ignore lips.
Even today as an adult, if I don't know the actors and I watch a French dub, I am not able to instantly tell it is a dub. I have to watch carefully for at least 10-20 seconds to make a guess. Maybe this speaks more of the fact that French dubs are actually really well made (they will choose translations so that French word cadence matches the English word cadence). On the other hand I have watched other dubs, especially non-English movies dubbed in English, where it's blatant that they were dubbed and it is very distracting.
In any case, I always get a kick when I tell my American friends that I watched US content dubbed all my childhood and it never bothered me :-)
[+] [-] ahartmetz|11 months ago|reply
Extreme example: children say "maMA" (strongly stressed on the last syllable, opposite of German) in dubbed French movies. It's actually a little annoying, haha.
[+] [-] pingou|11 months ago|reply
I'm french and I'm terrible at understanding speech when in a noisy environment for example. But I didn't grow up with tv so that's probably not it.
[+] [-] hiisukun|11 months ago|reply
For example, I will now look up the UFO Sweden show to see if I might like to watch it -- because I discovered it through here.
I have tried subscribing to a few different regular streaming services, but none seemed to work even if I pointed them in the right direction. I'm really not particular about which country of origin the production has, so long as it's "the best" or "very popular" in recent times from that place, it's worth me checking the genre and style to see if it piques my interest.
Any advice on this? [ nb. I have a similar issue with podcasts! ]
[+] [-] cess11|11 months ago|reply
[+] [-] dpig_|11 months ago|reply
[+] [-] fifilura|11 months ago|reply
In particular the SMHI security guards "stället måste outoymmas, det finns en bomb i huset".
[+] [-] chrononaut|11 months ago|reply
(I have attempted to study Swedish briefly, but I feel like I need to do more sustain discovery of content to consume in order to keep me engaged long term)
[+] [-] tokai|11 months ago|reply
[+] [-] snug|11 months ago|reply
[+] [-] panzagl|11 months ago|reply
https://youtu.be/J8P3vZo-0Hc?t=169
[+] [-] cratermoon|11 months ago|reply
[+] [-] cwmma|11 months ago|reply
1. https://www.youtube.com/watch?v=gnEIeVWLtbU&pp=ygUOaWNlIGljZ...
[+] [-] bbstats|11 months ago|reply
https://youtu.be/PTngv5MmtXo?si=cCQcYtRaja5pPP6a
[+] [-] gruez|11 months ago|reply
https://youtu.be/PTngv5MmtXo?t=105
They're admittedly pretty cagey with the parts they're willing to show, but the parts they did show look fine to me. Are you referring to the parts when they showed the 3d mask overlay on top of the characters?
[+] [-] ericmcer|11 months ago|reply
Seems legit lol, nothing to brush under the rug definitely.
[+] [-] akgoel|11 months ago|reply
[+] [-] numpad0|11 months ago|reply
[+] [-] ks2048|11 months ago|reply
https://www.vanityfair.com/hollywood/story/the-brutalists-ai...
[+] [-] vodou|11 months ago|reply
But apparently IMDb does not agree: "The Unthinkable", IMDb rating 6.0 "Watch the Skies", IMDb rating 6.4
[1] https://www.imdb.com/title/tt5227746
[+] [-] timewizard|11 months ago|reply
Did they literally ask a machine "visually dub subtitles into this movie file for me please" and get a result? No? They didn't use AI then. They used a purpose built model in a purpose built tool to perform a single algorithmic function. how that algorithm works or what technologies it's similar to are not relevant.
/curmudgeon
[+] [-] shermantanktop|11 months ago|reply
[+] [-] bdhcuidbebe|11 months ago|reply
And inference isnt? Explain your reasoning pls.
[+] [-] namaria|11 months ago|reply
[+] [-] dudus|11 months ago|reply
It sucks it will kill a large field for actors, but AI has the capacity to be much better with voices and performances that mimic the actors.
[+] [-] CaptainFever|11 months ago|reply
[+] [-] sounds|11 months ago|reply
[+] [-] ano-ther|11 months ago|reply
[+] [-] hulitu|11 months ago|reply
So, a new version of the Python's "Hungarian phrasebook" all over again.
[+] [-] yapyap|11 months ago|reply
What WOULD bother me is AI artifacts in a movie I paid to see in theaters.
[+] [-] ks2048|11 months ago|reply
Only a matter of time, for better or worse, until translation, synthesis, and visual modification all done with AI.
[+] [-] CaptainFever|11 months ago|reply
> Notably, the original actors recorded their own dialogues in English in a sound booth — Flawless AI's technology merely altered the movements of their lips in the movie.
[+] [-] nh23423fefe|11 months ago|reply
[+] [-] tantalor|11 months ago|reply
[+] [-] iancmceachern|11 months ago|reply