The strange new world of being a deepfake actor(technologyreview.com)
technologyreview.com
The strange new world of being a deepfake actor
https://www.technologyreview.com/2020/10/09/1009850/ai-deepfake-acting/
17 comments
Everyone hates DRM but one day DRM will be the only way to verify a video is legit.
You can very well digitally sign a video, DRM is not needed.
Can you sign it with the aljazeera key in your keychain?
No? But aljazeera can sign their content with their private key. That's how signing works...
Sure, but aljazeera can sign a deepfake with their key just as readily as a real video, the signature does not ensure the realness of the video - it can only establish a part of it's "chain of sources".
And that's the same with DRM? Unless I am missing something.
But are you going to trust aljazeera's signature on a [potential] deepfake of an American politician as the only source of truth about whether that video is real?
That isn't what signing does. Signing attests to origin not authenticity.
In a technical sense, signing only attests that a specific key pair was used to process the data. How that key pair is associated to its handler is a social function. If the signer claims authenticity, the general population will accept the claim, regardless of whether the claim has technical merit.
I'd trust more than a deepfake TicTok video reposted by FoxNews/CNN.
We can do a full "Video SecureBoot" thing, where the camera sensor signs stuff at the lowest level and every other level adds additional signatures, so you know the full chain of what happened (what cam took what part of the final video, what software edited, and who did the editing).
Nothing in that scheme requires "rights management".
Ultimately still it comes down to trust: do you trust the org who put out the video, do those people trust their subcontractors and (freelance) videographers, the hardware and software that was used, etc
In some cases this might help a little. But in the end misinformation and outright lies will still spread, be it deepfakes or be it just the written word. QAnon is relatively successful without the need to deepfake (they will just do a few meme pictures), so is the Flat Earth movement, and Alex Jones (US not UK) has still a following too, for example.
Nothing in that scheme requires "rights management".
Ultimately still it comes down to trust: do you trust the org who put out the video, do those people trust their subcontractors and (freelance) videographers, the hardware and software that was used, etc
In some cases this might help a little. But in the end misinformation and outright lies will still spread, be it deepfakes or be it just the written word. QAnon is relatively successful without the need to deepfake (they will just do a few meme pictures), so is the Flat Earth movement, and Alex Jones (US not UK) has still a following too, for example.
Several years ago I researched and worked on this concept in my spare time in the hope of creating an alternative to DRM and an alternative revenue model.
Instead of forcing users to use authorized devices and checking if they are allowed to consume the content I wanted to encourage users to desire authentic digital content. This would have involved a container capable of holding chains of signatures, PKI, and players with a feature similar to the padlock in browsers.
Authentic for pay content would have contained unique signatures that acted as a watermark.
If someone ilegally shared the signed version it could be traced back and action could be targeted. The goal was not to make it impossible to pirate content (I find it foolish to try to prevent this) but to make unsigned content undesirable due to uncertain authenticity. The same way authentic colectables are desirable due to their authenticity. A forgery of a famous painting is worth a lot less than the original if you know it is fake even if it is a perfect copy. A reissue of a first edition comic book is worth less even if the content is the same.
In some markets 3C marketing works better than 4P marketing and this results in a circular symbiotic relationship between creator and consumer rather than a hierarchical antagonic relationship.
This also included a way of remixing content that allowed distribution of the original along with the alterations. A sort of open source for media. Think of distributing your Blender/Premiere/ After Effects/etc. project file instead of the rendered result. It would have also been possible to share the alterations without the original and let the user provide their own copy if the original. This was based on a very wrong understanding of what is a derivative from a legal pov.
Alas, this was a foolish dream in my youth, but it was a vision of what media formats would have been like in a parallel world where authenticity was worth more than monopoly over distribution. A world where plagiarism would have been a bigger crime instead of copyright infringement.
Authentic for pay content would have contained unique signatures that acted as a watermark.
If someone ilegally shared the signed version it could be traced back and action could be targeted. The goal was not to make it impossible to pirate content (I find it foolish to try to prevent this) but to make unsigned content undesirable due to uncertain authenticity. The same way authentic colectables are desirable due to their authenticity. A forgery of a famous painting is worth a lot less than the original if you know it is fake even if it is a perfect copy. A reissue of a first edition comic book is worth less even if the content is the same.
In some markets 3C marketing works better than 4P marketing and this results in a circular symbiotic relationship between creator and consumer rather than a hierarchical antagonic relationship.
This also included a way of remixing content that allowed distribution of the original along with the alterations. A sort of open source for media. Think of distributing your Blender/Premiere/ After Effects/etc. project file instead of the rendered result. It would have also been possible to share the alterations without the original and let the user provide their own copy if the original. This was based on a very wrong understanding of what is a derivative from a legal pov.
Alas, this was a foolish dream in my youth, but it was a vision of what media formats would have been like in a parallel world where authenticity was worth more than monopoly over distribution. A world where plagiarism would have been a bigger crime instead of copyright infringement.
Breaking: Encryption Inadequate Without Trust
Except DRM is even less well suited for that task than it is for its original task. The threat model is even more ridiculous.
[deleted]
The last video in the article, with the British prime minister is very good. This is going to be interesting going forward. Anyone could start denying anything they said on camera and claim deep fake. A lot of people seem to not want to really try to discern some truth in what is being said by politicians, so deep fakes as a tool could be useful even by just claiming they are used.
As a joke, it will also make films, such as the pilot of Black Mirror, pretty undramatic: "Was it really the British prime minister who had intercourse with a pig, or was it just a deep fake?" :)))
Photoshop and 'morphing' did not lead to deluge of claims and counter-claims around fake pictures. Why?
Do society and human behavior adopts to new reality?
Do society and human behavior adopts to new reality?
What? It absolutely did!
I believe that this has been widespread on social media but do you know of any examples where reputable media outlets were fooled by a Photoshop? Not saying I don't believe you but I watch plenty of news and haven't personally seen much of this.
National Geographic moved the Pyramids for a cover photo in '82.
https://www.alteredimagesbdc.org/national-geographic/
Granted that's a case of a reputable media outlet fooling their readers by Photoshop, but it's arguably part of the problem.
https://www.alteredimagesbdc.org/national-geographic/
Granted that's a case of a reputable media outlet fooling their readers by Photoshop, but it's arguably part of the problem.
I think a case could be made that Nat Geo certainly took a hit to their reputation:
"The manipulation damaged the magazine’s credibility. Tom Kennedy, who became National Geographic’s Director of Photography after the incident, stated: “We no longer use that technology to manipulate elements in a photo simply to achieve a more compelling graphic effect. We regarded that afterwards as a mistake, and we wouldn’t repeat that mistake today.”
[https://www.alteredimagesbdc.org/national-geographic/]
In any case, I was certain it had happened but this isn't exactly compelling evidence that it's a widespread problem. A much more recent example would help your case. Also, (as you mentioned) this is a case of an outlet fooling people with their own fakes, not an outlet getting fooled by a fake which is more what I was referring to.
"The manipulation damaged the magazine’s credibility. Tom Kennedy, who became National Geographic’s Director of Photography after the incident, stated: “We no longer use that technology to manipulate elements in a photo simply to achieve a more compelling graphic effect. We regarded that afterwards as a mistake, and we wouldn’t repeat that mistake today.”
[https://www.alteredimagesbdc.org/national-geographic/]
In any case, I was certain it had happened but this isn't exactly compelling evidence that it's a widespread problem. A much more recent example would help your case. Also, (as you mentioned) this is a case of an outlet fooling people with their own fakes, not an outlet getting fooled by a fake which is more what I was referring to.
That's a somewhat different case though. That gets into what degree of image manipulation for aesthetic purposes is acceptable for a given use. Because any photograph reflects decisions about composition, lens, film (at the time)/color temperature/B&W vs. color, and so forth. And what you might choose to do with a vacation picture (or, for that matter, a fine art poster) isn't necessarily acceptable for many publications .
You also have the photo illustration that Time did of OJ Simpson which was about photo manipulation to fit with a narrative tone. Of course, photo selection could have the same effect.
You also have the photo illustration that Time did of OJ Simpson which was about photo manipulation to fit with a narrative tone. Of course, photo selection could have the same effect.
Why is voice audio always so hard to get right? Seems like it should be easier than video because the data rate is lower than video but computer generated audio has always been terrible.
My thought is that human speech contains a huge amount of delicate, not conciously noticable quailites that we are innately attuned to pick up on but find hard to explain explicitly.
This is also true of visual facial expressions, but deep fakes know to avoid close ups on the face otherwise the deceit would be apparent, and we are accustomed to seeing faces from afar. Audio on the other hand, we just dont accept perceiving it badly: We'll ask the person to speak louder and clearer, turn up the volume, come closer etc.
This is also true of visual facial expressions, but deep fakes know to avoid close ups on the face otherwise the deceit would be apparent, and we are accustomed to seeing faces from afar. Audio on the other hand, we just dont accept perceiving it badly: We'll ask the person to speak louder and clearer, turn up the volume, come closer etc.
Maybe because video is 2D, so it doesn’t seem real anyway, but audio is very close to real-life experience, so it is easier to notice the difference.
Either a possibility or a tangent: video is less time sensitive (think 24 fps film) than audio, which in turn is less time sensitive than haptic feedback.
(Sentiment du fer may seem like a strange idea to us, but in disciplines where 40 ms is important, it may help to base reactions on senses which are more time sensitive than the eyes.)
(Sentiment du fer may seem like a strange idea to us, but in disciplines where 40 ms is important, it may help to base reactions on senses which are more time sensitive than the eyes.)
I wait for cg chatacters to animate on a text script by using move/acting library, then it will be easy to shoot your own dynamic 3d movies by feeding a novel into the app and deepfaking the resulting cg scene. Combined with neural links, even not so good ones, that would be an entire new universe of... this term has yet to be invented. And you guys are thinking about deceitful politicians, as if they were relevant in it. The real world that these dinosaurs cling to is melting away faster than they shake their finger on camera.
Sounds like an enhanced XtraNormal. I can't wait to see the "MongoDB is Web Scale" expanded universe on the silver screen.
You could use a language model like gpt-3 for the novel part. In fact, I think that will be the first part of the pipeline that will be of sufficient quality.
How so? I thought the poster meant that they are the ones who will be writing the novels. Or did I misunderstand?
I don't really think you could rely 100% on AI for creative things like novels. A fundamental problem with AI (in my humble opinion) is that it simply doesn't have a grasp on reality. Unless if you constantly take care to guide it, it will go off the rails very fast.
I don't really think you could rely 100% on AI for creative things like novels. A fundamental problem with AI (in my humble opinion) is that it simply doesn't have a grasp on reality. Unless if you constantly take care to guide it, it will go off the rails very fast.
I’ve seen movies go off the rails fast. You might get a few accidental cult classics out of it.
Good article, but I did a double take when it said "the director wanted to protect the activists so they used deep fakes to hide their faces, here's the before and after". What's the point of hiding the face if you're going to show the before?!
As I understood it, that particular activist had gone public by that point.
Doppelgangers have existed since forever. I don't understand why this is such a big deal now.
Here is a random agency where you can book real-life doppelgangers:
http://www.lookalikes.info/
Here is a random agency where you can book real-life doppelgangers:
http://www.lookalikes.info/
Handwritten codexes have existed for long, why is the printing press such a big deal?
A subversive actor only needs a lookalike of one or two people. And it's not like we can't duplicate a video of a doppelganger. So the analogy is lost on me.
The new tech allows to do the same on a massive scale. Yes Stalin had people working for him who would manually erase from photos the people he didn't like any more. It was possible. Photoshop's content-aware fill and various deep learning inpaining algorithms can do that at scale with a fraction of the effort.
The same way, yes it was possible to hire a lookalike, but a) the deepfake will look more similar to the target person than any lookalike can and b) you don't need to find a needle-in-a-haystack lookalike to generate a fake video of anyone you want.
Or, "the post has been carring messages to people for centuries, why is the Internet such a big deal?"
The same way, yes it was possible to hire a lookalike, but a) the deepfake will look more similar to the target person than any lookalike can and b) you don't need to find a needle-in-a-haystack lookalike to generate a fake video of anyone you want.
Or, "the post has been carring messages to people for centuries, why is the Internet such a big deal?"
You make a glib attempt at a parallel that's not relevant to the topic in the article. The point of this article isn't that we now have a "mechanical" process to make faking easier. That's been known for awhile. The headline specifically calls out acting for the purpose of deepfakes, but the article goes on to discuss the troubles deepfakes can cause.
It's certainly arguable that doppelgangers made "deepfakes" easier in the past, but only easier to those who could hire the actor. And that this technology makes it so that hiring the doppelganger is unnecessary, but it's by no means easier like the printing press made copying books easier.
It's certainly arguable that doppelgangers made "deepfakes" easier in the past, but only easier to those who could hire the actor. And that this technology makes it so that hiring the doppelganger is unnecessary, but it's by no means easier like the printing press made copying books easier.
The real actor in a deepfake doesn't have to look much like the target person. The pool of usable actors thus explodes by many many orders of magnitude. Hence my comparison.
>I don't understand why this is such a big deal now
because, as the article points out, the actor in the deepfake isn't going to actually act out what the deefake is later going to produce. So this makes creating nefarious content possible by letting the actor read something harmless and then replacing voice and appearance with say, a scam or whatever else. You could hire an actor to read something random and then use deepfakes to make it look like a kidnapping to extort someone or whatever.
If you hire an actual human for the entire thing you'd have to clue them in, and pretty much every commercial business is probably going to not participate in something criminal. Deepfakes probably could scale one innocent performance up to thousands of potential malicious ones.
You know these scams where old people are being called and told that their grandson had an accident and you need to wire them money? Imagine that with video footage and 100% fidelity.
because, as the article points out, the actor in the deepfake isn't going to actually act out what the deefake is later going to produce. So this makes creating nefarious content possible by letting the actor read something harmless and then replacing voice and appearance with say, a scam or whatever else. You could hire an actor to read something random and then use deepfakes to make it look like a kidnapping to extort someone or whatever.
If you hire an actual human for the entire thing you'd have to clue them in, and pretty much every commercial business is probably going to not participate in something criminal. Deepfakes probably could scale one innocent performance up to thousands of potential malicious ones.
You know these scams where old people are being called and told that their grandson had an accident and you need to wire them money? Imagine that with video footage and 100% fidelity.
So if you're framed by deepfake the only way to prove it's not you is to come up with film of you elsewhere. And so you're going to film yourself wherever you are to mitigate being framed...?
The other probably more appealing use is to have a credible figure (say maybe Greta Thunberg) say she was just kidding about climate change and global warming. Fake news is going to have a party. It's going to get ever more difficult to tell what's true and what isn't.
The other probably more appealing use is to have a credible figure (say maybe Greta Thunberg) say she was just kidding about climate change and global warming. Fake news is going to have a party. It's going to get ever more difficult to tell what's true and what isn't.
No need to film yourself. You just got to make sure your deepfaked alibi is better.
I wonder if in the future it's not about being able to afford better lawyers, but better fakes.
I wonder if in the future it's not about being able to afford better lawyers, but better fakes.
> You just got to make sure your deepfaked alibi is better.
This could be a better pivot for Stadia than cloud gaming. Cloud deep fakes for everyone.
This could be a better pivot for Stadia than cloud gaming. Cloud deep fakes for everyone.
This page opened not one, not two, not three, but four pop-ups that I needed to close to get to the article.
1. cookie acceptance
2. subscribe to mailing list
3. subscribe to technology review
4. some ad that overtook half of the screen
Do they want people actually read this website?
1. cookie acceptance
2. subscribe to mailing list
3. subscribe to technology review
4. some ad that overtook half of the screen
Do they want people actually read this website?
Why would they want people to read the website? It’s a hard thing to measure and hard to monetize. Most want you to visit a website, get tracked and served an ad accordingly, follow the ad and go buy something. If that’s not happening, the second best thing is to get your communication info so that they can try again.
Reading the website often can be considered harmful as you may not share it if the article is not as interesting as title that took you there.
Reading the website often can be considered harmful as you may not share it if the article is not as interesting as title that took you there.
You and I live in totally different worlds and I pity your entire internet browsing experience.
I had to open up an incognito browser to see what you were talking about. In my normal browsing experience I only got the thing on the bottom half. And perhaps there is a possibility that I already had managed the cookies in the past.
I had to open up an incognito browser to see what you were talking about. In my normal browsing experience I only got the thing on the bottom half. And perhaps there is a possibility that I already had managed the cookies in the past.
Yeah, it might be because you already clicked all those 4 pop-ups in the past.
Thankfully, I can still read their article with javascript blocked (something that stops working on more and more sites), so I wasn't hit by that.
I browse with js disabled, and I had none of those issues.
Just wanted to record that Glyph Lefkowitz pointed out like twenty years ago that we would one day have e.g. new Indiana Jones movies [with realistic pseudo-Harrison Ford].
Sounds like Diamond Age should be required reading - scifi is too real.
A really good demonstration of this would be to redo the trumpbiden debates with subtle changes in what they were saying.
This article is just an odd headline hook to discuss deep fakes. "Being" a deep fake actor is simply being the source media driving the VFX character. As an authoritative source on the topic, this is about as shallow as the rest of the pop culture articles on the topic - which is ankle deep.
To me there were "tells" - the mouths in particular looked unnatural. I wonder if we will soon see competing technology: deep-fake detectors trained by deep fake creators?
> This is how Lewis D. Wheeler, a Boston-based white male actor
The fact that this framing is so normalized to just be thrown out in a mostly technical article with no other explanation is insidious and insane. There's video of the guy right above. It's very clear what his skin color and gender are. And it's totally off topic.
At a minimum if you are going to do this, it should be done for every single person mentioned consistently. Why don't we get Nixon's gender and race or Francesca Panetta and Halsey Burgund genders and races? And of course when you have the choice between doing this consistently for everyone or not for anyone it becomes clear that it would be way over the top if done consistently and in reality is inappropriate in all cases.
It gets into Noam Chomsky's manufacturing consent arguments in my mind. Everything is a meaningful editorial decision. This IS a political statement. By including this you are implying it would have been undescriptive and impproper not to include whereas the opposite is the case.
And yes, I know it's off topic but I think it is important to call out nonsense when you see it lest it be normalized. If you want your article to be discussed instead of prompting a culture war, then don't bait a culture war.
The fact that this framing is so normalized to just be thrown out in a mostly technical article with no other explanation is insidious and insane. There's video of the guy right above. It's very clear what his skin color and gender are. And it's totally off topic.
At a minimum if you are going to do this, it should be done for every single person mentioned consistently. Why don't we get Nixon's gender and race or Francesca Panetta and Halsey Burgund genders and races? And of course when you have the choice between doing this consistently for everyone or not for anyone it becomes clear that it would be way over the top if done consistently and in reality is inappropriate in all cases.
It gets into Noam Chomsky's manufacturing consent arguments in my mind. Everything is a meaningful editorial decision. This IS a political statement. By including this you are implying it would have been undescriptive and impproper not to include whereas the opposite is the case.
And yes, I know it's off topic but I think it is important to call out nonsense when you see it lest it be normalized. If you want your article to be discussed instead of prompting a culture war, then don't bait a culture war.
1. The writer often does not know what other media is going to be edited into the article, especially if it also runs in print.
2. Much of the article talks about what factors are important and what aren't for a "deep fake actor." Skin color and gender seem absolutely relevant. (I don't necessarily disagree it could have been mentioned in other places as well.)
2. Much of the article talks about what factors are important and what aren't for a "deep fake actor." Skin color and gender seem absolutely relevant. (I don't necessarily disagree it could have been mentioned in other places as well.)
Regarding point 2. That's totally valid if they come out and say why it matters. But they totally fail to connect those dots making it unclear if they are making that point or not.
Further down it says:
>For the visuals, Canny AI specializes in video dialogue replacement, which uses an actor’s mouth movements to manipulate someone else’s mouth in existing footage. The actor, in other words, serves as a puppeteer, never to be seen in the final product. The person’s appearance, gender, age, and ethnicity don’t really matter.
So I could argue it either way. If the gender and ethnicity didn't matter in the Nixon case, the writer/editor should probably have indicated that on the first mention. (On the other hand, they also wanted someone who sounded a bit like Nixon which would tend to lead you to a white male.)
>For the visuals, Canny AI specializes in video dialogue replacement, which uses an actor’s mouth movements to manipulate someone else’s mouth in existing footage. The actor, in other words, serves as a puppeteer, never to be seen in the final product. The person’s appearance, gender, age, and ethnicity don’t really matter.
So I could argue it either way. If the gender and ethnicity didn't matter in the Nixon case, the writer/editor should probably have indicated that on the first mention. (On the other hand, they also wanted someone who sounded a bit like Nixon which would tend to lead you to a white male.)
While I think it could have been structured better, I interpreted it as the baseline against which the later instance of the subject and actor being different races is contrasted.
> This is how Lewis D. Wheeler, a Boston-based white male actor, found himself holed up in a studio for days listening to and repeating snippets of Nixon’s audio. There were hundreds of snippets, each only a few seconds long...
Wheeler’s job was to re-record each snippet in his own voice, matching the exact rhythm and intonation. These little bits were then fed into Respeecher’s algorithm to map his voice to Nixon’s.
It would be nice to hear what the actor was able to produce directly, before it was "mapped" to be Nixon. With the amount of effort that was spent, I'd presume that there are actors good enough to create a convincing fakes even without the post-processing. Are there any before-and-after samples available?
It would be nice to hear what the actor was able to produce directly, before it was "mapped" to be Nixon. With the amount of effort that was spent, I'd presume that there are actors good enough to create a convincing fakes even without the post-processing. Are there any before-and-after samples available?
[deleted]