New Software Tech Allows Believable Manipulation of Faces in Videos

[John Keefer]

New Software Tech Allows Believable Manipulation of Faces in Videos


As if it wasn't hard enough already to believe what you see and hear on the Internet, new software is being developed that allows you to manipulate the faces and mouth movements of people in existing videos.

Imagine, if you will, a broadcast of a prominent world leader giving a live address being watched by millions. This one-time dove has suddenly become a hawk, announcing war and totalitarian policies. Little would you know that the leader's speech has been altered by software that can do real-time manipulation of the face and mouth by a voice actor saying the words.

Sound far-fetched? Not with some new "Face2Face" technology being developed by researchers from the University of Erlangen-Nuremberg, Max-Planck-Institude for Informatics, and Stanford University. The technical video shows the "target actor" - in this case the politician of your choice - rendered with a neutral expression. Then the source actor could say any words or offer any expressions of their choice. Right now, the technology works on existing YouTube videos.

This is the same team that did real-time expression transfer [http://graphics.stanford.edu/~niessner/thies2015realtime.html] just last year. The project is looking at real-world applications that would allow a translator to give a guest speaker the appearance of speaking in the native tongue of an audience.

Obviously, it is still a work in progress, but the potential is fascinating, if not downright scary. We all know that programs like this could easily be used for more nefarious means, right?

Source: Stanford University [http://www.graphics.stanford.edu/~niessner/thies2016face.html]

Permalink

 

[DemomanHusband]

That's legitimately kind of spooky! I suppose with the existence of FaceRig and the like, this wasn't too far off, although I must admit that this makes me worry for the future. If things get to the point where any talking point, any quip can be mimicked with a long enough silence in a video to fill in... Well, we'll see what happens.

Gotta admit though, this technology is pretty nifty.

 

[Lightknight]

Whoa, so they can now dub movies and have it still look like the actor is saying it. Pretty cool.

With voice modulating technology though, this produces some scary possibilities where you can spoof videos of people and try to pass them off as authentic.

 

[Baresark]

That is pretty awesome. Part of me is like, "we can longer trust video", then another part of notices that no one trusts video any longer anyway.

 

[IamLEAM1983]

Two things come to mind: for one, this would solve the single problem of dubbed movies; which is ill-fitting labial movements in relation to what's heard.

Secondly, where was this technology when Forrest Gump was being shot? It would've been a perfect fit for the movie!

 

[RealRT]

So now we can have dead actors star in movies?

 

[sorsa]

So now we can have dead actors star in movies?

I had a discussion today with my brother about how sad it will be when legends like Schwarzenegger and Stallone are gone.

Now we just need to prop up their corpses and the software will take care of the rest.

 

[weirdee]

So now we can have dead actors star in movies?

I had a discussion today with my brother about how sad it will be when legends like Schwarzenegger and Stallone are gone.

Now we just need to prop up their corpses and the software will take care of the rest.

Actually, all we need are people who vaguely have the same body type, and just superimpose somebody else's face on them.

That is, until we get those servomotors that can move realistically, and then we can start talking about corpse puppets.

 

[sorsa]

So now we can have dead actors star in movies?

I had a discussion today with my brother about how sad it will be when legends like Schwarzenegger and Stallone are gone.

Now we just need to prop up their corpses and the software will take care of the rest.

Actually, all we need are people who vaguely have the same body type, and just superimpose somebody else's face on them.

Ok.. but that's not nearly as fun as having flesh marionettes for actors and you know it.

 

[pearcinator]

I get a real uncanny valley vibe from the video. It's still not quite right, close but you can tell it's fake and as a result it looks like a dead body making faces.

 

[Lightknight]

I get a real uncanny valley vibe from the video. It's still not quite right, close but you can tell it's fake and as a result it looks like a dead body making faces.

That's because Putin never looks natural when he smiles.