[sleepydog]

I noticed Netflix doing this while watching this series: https://about.netflix.com/news/beyond-goodbye-premieres-nove...

It was so jarring I couldn't watch it and switched to Japanese with subtitles. Their mouths looked so stiff and unnatural and out of sync with their expressions that all me and my wife could talk about was how bad it looked.

[lambdaone]

You can see a lot of open mouth/tongue stuff being skipped. Dealing with the tongue and the inside of the mouth is a huge problem with this sort of visual dubbing. Using traditional techniques, you can model teeth and gums as rigid bodies, and faces as rubber sheets (to first approximation), but tongues, for which you typically have no visual reference in any given shot, are much more difficult to model, and continuously, subtly, on the move. "AI" is the general answer to this problem nowadays, but even ML-based systems struggle to deal with the tongue issue while trying to reconcile visual appearance with animation fidelity.

[basch]

But the actors re-recorded their dialog in English in the studio. Why wouldn’t you video record their mouths? In fact, the old punch the mouth out and pop the new one in behind the whole can be surprisingly effective sometimes. I would think having an actual visual reference and doing more of a deepfake on the lips should provide excellent results, vs synthetic lip motion to match the audio only.

[SirFatty]

Yes, Kung Pow is a great example of how jarring it can be! :-D

[m463]

It's funny how older movies like that are unintentionally prescient.

Like the Kentucky Friend Movie "Eyewitness News" segment and smart TVs.

[diftraku]

"YOU WANT FRIES WITH THAT?"

[js2]

> Their mouths looked so stiff and unnatural and out of sync with their expressions that all me and my wife could talk about was how bad it looked.

Everything old is new again. Classic Italian films were almost always dubbed by their original actors, to the point that Fellini sometimes didn't even write portions of the dialog till after filming was done. The actors just said random stuff during filming. Originally this was mainly a technology limitation (sets were too noisy), then it became a cost-savings measure and cultural tradition to how films were made in Italy.

I always find it a bit of a distraction when I haven't watched an Italian film in a while, but eventually get used to it.

[petters]

Dubbing by the original actors is very common when making movies. But usually the dialog is the exact same so it’s not noticed

[chedabob]

It was quite jarring in Amazon's Citadel Diana. The voices were ever so slightly out of sync with the lip movements, and the audio sounded like the studio recording hadn't been processed to match the environment.