Nvidia uses GAN for video compression(youtube.com)
youtube.com
Nvidia uses GAN for video compression
https://www.youtube.com/watch?v=NqmMnjJ6GEg&ab_channel=NVIDIADeveloper
3 comments
This and other ways of using ML for video compression (like DLSS) scare me a bit because it kinda screws with reality. Like, with MPEG compression, you just see less detail when there's less bandwidth and you know exactly what you're missing. But we seem to be moving to methods where instead of removing the missing data, we fill it in with what should most likely be there. There's something that just seems wrong with that.
It will be fine as long as codecs still encode difference between model prediction/ML hallucination and ground truth. We dont need another Xerox compression fiasco http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...
> instead of removing the missing data, we fill it in with what should most likely be there
Just like the blind spot caused by your optic nerve!
Just like the blind spot caused by your optic nerve!
IMO this is a fantastic use of this technology. The bandwidth is low enough to enable videoconferencing on a dial-up connection. I only wonder how big the model is - gigabytes I bet.
The article state that it does send over at least 1 key-frame.
That might be used to find coefficients in a much smaller model, similar to how Active Appearance Models approximate a lot of faces with a relatively small model.
Existing discussion: https://news.ycombinator.com/item?id=24694565