VP8 vs H.264 – Which One is Better?(gist.github.com)
gist.github.com
VP8 vs H.264 – Which One is Better?
https://gist.github.com/4645784
60 comments
Testing with a single not very representative clip is a poor idea, See the comment on the github: https://gist.github.com/4645784#comment-756444
>not very representative clip
I chose the clip based on what Dark_Shikari (x264 developer) had to say about it[1]:
It shouldn't bias too heavily towards any one encoder like many of the other standard test clips will:
a. It's relatively high motion, so it won't bias heavily against encoders without B-frames or qpel (as, say, mobcal does).
b. It's not so high motion that it would cripple video formats that don't support motion vectors longer than 16 pixels (e.g. Theora).
c. It's not something that benefits an unreasonably large amount from some of x264's algorithms (which is why I picked this and not parkrun).
[1] http://forum.doom9.org/showthread.php?t=154430
I could have done multiple test encodes, sure, but the problem in this case was that downloading several gigabytes of raw source material isn't exactly instant. And even if I tested with multiple clips, I doubt the conclusion would be that much different.
I chose the clip based on what Dark_Shikari (x264 developer) had to say about it[1]:
It shouldn't bias too heavily towards any one encoder like many of the other standard test clips will:
a. It's relatively high motion, so it won't bias heavily against encoders without B-frames or qpel (as, say, mobcal does).
b. It's not so high motion that it would cripple video formats that don't support motion vectors longer than 16 pixels (e.g. Theora).
c. It's not something that benefits an unreasonably large amount from some of x264's algorithms (which is why I picked this and not parkrun).
[1] http://forum.doom9.org/showthread.php?t=154430
I could have done multiple test encodes, sure, but the problem in this case was that downloading several gigabytes of raw source material isn't exactly instant. And even if I tested with multiple clips, I doubt the conclusion would be that much different.
Because of texture this clip benefits enormously from 8x8 transforms (as well as substantially from an activity masking aware encoder). On an intra frame in prior testing Theora did enormously better than VP8 on this clip for these reasons. If your test was to compare an intraframe between vp8 / baseline h264 / and Theora, you would have concluded Theora was the best by a wide margin. But this would be an erroneous conclusion.
And sure, perhaps you'd get the same result on other clips. Over high-profile H264 the only obvious format feature that come to mind that could really let VP8 get ahead are the 'truemotion' intra-predictor and creative use of the synthetic reference frame (though I suppose the vp8 developers might have other suggestions) and I'd expect those features to only be big wins on a small number of clips so it wouldn't be hard to miss the cases where VP8 really shines over high profile h264.
But you (or I) could have said that without doing the test at all, and there would be 100% fewer clueless people going around claiming that something was proven here that wasn't. Your opinion (or mine) is a fine thing, but it's not proper to launder an opinion as fact by dressing it up in an inadequate test.
And sure, perhaps you'd get the same result on other clips. Over high-profile H264 the only obvious format feature that come to mind that could really let VP8 get ahead are the 'truemotion' intra-predictor and creative use of the synthetic reference frame (though I suppose the vp8 developers might have other suggestions) and I'd expect those features to only be big wins on a small number of clips so it wouldn't be hard to miss the cases where VP8 really shines over high profile h264.
But you (or I) could have said that without doing the test at all, and there would be 100% fewer clueless people going around claiming that something was proven here that wasn't. Your opinion (or mine) is a fine thing, but it's not proper to launder an opinion as fact by dressing it up in an inadequate test.
>If your test was to compare an intraframe between vp8 / baseline h264 / and Theora, you would have concluded Theora was the best by a wide margin.
But it wasn't. I was comparing the visual quality of the whole video, and provided the full encoded clips for people to download and compare for that reason.
I am willing to do further test encodes, but have no interest in doing something like encoding all 28 HD test clips available on derf's test clip page[1], since as a purely visual comparison, especially with the actual encodes, it would be incredibly exhausting.
EDIT: I added a notice about the downsides of single clip comparison to the top of the post.
[1] http://media.xiph.org/video/derf/
But it wasn't. I was comparing the visual quality of the whole video, and provided the full encoded clips for people to download and compare for that reason.
I am willing to do further test encodes, but have no interest in doing something like encoding all 28 HD test clips available on derf's test clip page[1], since as a purely visual comparison, especially with the actual encodes, it would be incredibly exhausting.
EDIT: I added a notice about the downsides of single clip comparison to the top of the post.
[1] http://media.xiph.org/video/derf/
I have no interest in doing something like encoding all 28 HD test clips, since as a purely visual comparison, it would be incredibly exhausting.
Science is exhausting. If you're not working hard, then you're likely to miss the interesting (counter-intuitive) results. In fact, finding counter-intuitive results is the whole point of science. If the truth were intuitive, explanations wouldn't need testing.
Science is exhausting. If you're not working hard, then you're likely to miss the interesting (counter-intuitive) results. In fact, finding counter-intuitive results is the whole point of science. If the truth were intuitive, explanations wouldn't need testing.
One problem is that even if I found that VP8 performed very well at one or two particular clips (out of the 28 HD test clips available), I couldn't say for sure why that is the case. There seems to be no clear information on what clips benefit from what kind of features, and as I'm not an expert on video encoding technology, it'd be hard for me to deduce these things by myself. General conclusions could still be reached, obviously, but if I was going to such lengths it'd suck if I couldn't get more overall detailed results.
Anyway, I brought up the subject to some Xiph folks over at IRC. Maybe in the future the test clips will come equipped with more detailed information to help in testing. It'd also benefit smaller scale tests, since it'd allow one to identify possible biases more easily.
Anyway, I brought up the subject to some Xiph folks over at IRC. Maybe in the future the test clips will come equipped with more detailed information to help in testing. It'd also benefit smaller scale tests, since it'd allow one to identify possible biases more easily.
That was in 2005!
Given that people do hundreds of test encodes when they actually use things like x264, I think that if you want to say anything general about these encoders you have to do more than one comparison.
Given that people do hundreds of test encodes when they actually use things like x264, I think that if you want to say anything general about these encoders you have to do more than one comparison.
Agreed, a single video is not very representative and the results are still subjective. I would have liked these claims to be backed up by measurements like PSNR or VQM over multiple sources.
Let's see how Daala and VP9 work out. Something should break the domination of closed formats, and MPEG-LA obviously plan to extend their grip on the market with H.265, since patents on H.264 are going to expire. Introducing an updated format will start this whole patent lock from the beginning.
Closed formats haven't dominated since the heyday of RealMedia in the late 1990s.
Support for H.264 in the hardware is commonplace, while support for VP8 is limited to few new SoCs and is virtually absent in any video dedicated hardware. You don't see it as a domination of closed formats?
H.264 isn't exactly "closed" per se - the standard itself is freely available[1], which is why we have great free and open source encoders and decoders for it. What you mean by "closed" is most likely just "patent-encumbered", which it most certainly is and which affects anyone wanting to use it commercially (at the moment you can use it freely for non-commercial purposes on the internet, but this may or may not change in the future).
[1] http://www.itu.int/rec/T-REC-H.264-201201-I/en
[1] http://www.itu.int/rec/T-REC-H.264-201201-I/en
By closed I mean not available for free use and distribution, for example in open source projects which can't pay any royalties. May be saying free format would sound more clear, since you are right, closed is a bit ambiguous, as it can also mean a format without published specification which needs to be reverse engineered to be used.
Also, I don't think that H.264 forbids only commercial use. Can you freely distribute their decoders and encoders? What about putting them in hardware for non commercial use?
Also, I don't think that H.264 forbids only commercial use. Can you freely distribute their decoders and encoders? What about putting them in hardware for non commercial use?
The fine print of H264 says as a consumer you can implement the codec without paying a license fee for non commercial personal use. That does, however, mean all the free implementations like in FFMPEG are liable because since they are open sourced they are not restricted to consumer use.
Open source projects can't pay royalties. But people who use them can be forced to.
The idea that VP8/VP9 will gain traction and remain royalty free is laughable.
The idea that VP8/VP9 will gain traction and remain royalty free is laughable.
You keep bashing anything not h.26x in multiple threads. MPEG-LA threatened to assemble an anti-VP8 patent pool ages ago, and that has yet to materialize. Show us the goods, and give a really darn good reason why the ideals behind wanting a royalty-free codec don't matter.
> Open source projects can't pay royalties. But people who use them can be forced to.
That's right, and that's the reason to avoid any closed codecs.
> The idea that VP8/VP9 will gain traction and remain royalty free is laughable.
What's laughable? VP8 remains royalty free. VP9 will as well. In practice you can never guarantee that some submarine patent troll won't appear tomorrow to threaten you. But the same perfectly applies to H.264/H.265 so your argument is irrelevant, since such threat applies to virtually anything, but it doesn't mean one should stop innovating because of it.
That's right, and that's the reason to avoid any closed codecs.
> The idea that VP8/VP9 will gain traction and remain royalty free is laughable.
What's laughable? VP8 remains royalty free. VP9 will as well. In practice you can never guarantee that some submarine patent troll won't appear tomorrow to threaten you. But the same perfectly applies to H.264/H.265 so your argument is irrelevant, since such threat applies to virtually anything, but it doesn't mean one should stop innovating because of it.
DarkShikari (x264 dev) has blogged about VP8 a few times: http://x264dev.multimedia.cx/archives/category/vp8
"The first in-depth technical analysis of VP8" http://x264dev.multimedia.cx/archives/377
"The first in-depth technical analysis of VP8" http://x264dev.multimedia.cx/archives/377
His blog is very insightful. There is also another great post where he talks about how to cheat on video comparison tests ( http://x264dev.multimedia.cx/archives/472 ). One of the things brought up in there is for still frame comparisons. If the frame came from an I frame rather than a B* or P frame in the output video, the test will be unfair.
*VP8 doesn't have B frames, similar to the h.264 baseline profile.
*VP8 doesn't have B frames, similar to the h.264 baseline profile.
There's no doubting gasset-glaser's (DarkShikari) knowledge in these matters, but he is hardly an objective party.
Apart from being one of the developers of open source GPL licenced x264 encoder, he and the other developers are also licencing it for proprietary use (nothing wrong with that, I think it's a great way of making money out of open source development). This means that a open source royalty free codec like VP8 (and later VP9) could cut into their revenue stream.
I'm not saying that this affected the outcome of his test, but it's still context. Also his test back then was also criticized for only using one clip, here's another test with a larger set of comparisons:
http://qpsnr.youlink.org/vp8_x264/VP8_vs_x264.html
I can't vouch for the validity of this anymore than the validity of any other test.
Apart from being one of the developers of open source GPL licenced x264 encoder, he and the other developers are also licencing it for proprietary use (nothing wrong with that, I think it's a great way of making money out of open source development). This means that a open source royalty free codec like VP8 (and later VP9) could cut into their revenue stream.
I'm not saying that this affected the outcome of his test, but it's still context. Also his test back then was also criticized for only using one clip, here's another test with a larger set of comparisons:
http://qpsnr.youlink.org/vp8_x264/VP8_vs_x264.html
I can't vouch for the validity of this anymore than the validity of any other test.
That he has. My test clip choice was inspired by him[1], and I also link to another blog post of his[2] in the conclusion. That VP8 blog post is almost three years old now, though, and the comment I'm replying to in my article claimed that VP8 is better than H.264 "at this point". This is why I did my test with the latest and greatest encoders available for both formats today.
[1] http://forum.doom9.org/showthread.php?t=154430
[2] http://x264dev.multimedia.cx/archives/472
[1] http://forum.doom9.org/showthread.php?t=154430
[2] http://x264dev.multimedia.cx/archives/472
I think it would be interesting to do a follow-up comparison h264 vs VP8 for mobile. 1080p vbr targetting ~7 Mbps, 480p vbr targetting ~2.5 Mbps, etc.
It's important to explore the space a bit for discussion's sake.
It's important to explore the space a bit for discussion's sake.
I think it would be more interesting to compare VP9 with HEVC (h.265), I know the VP9 experimental branch can easily be checked out and tested, but I don't know of where to find HEVC encoders (although I'm sure they exist with some Google fu).
Obviously there are problems with testing unfinished implementations, as:
A) they are generally VERY slow given that there's little optimization during the development phase, and these next gen video codecs are more demanding than the previous generation
B) a great deal of the quality comes from fine-tuning during implementation, not from the specification. For example x264 is a 'best-in-class' implementation of h.264, readily beating many other implementations of the very same specification.
As such, this fine-tuning is not likely to exist until the codec implementations mature, which likely means that what is available now of HEVC/VP9 serves only as very rough estimates.
Would still interesting though :)
Obviously there are problems with testing unfinished implementations, as:
A) they are generally VERY slow given that there's little optimization during the development phase, and these next gen video codecs are more demanding than the previous generation
B) a great deal of the quality comes from fine-tuning during implementation, not from the specification. For example x264 is a 'best-in-class' implementation of h.264, readily beating many other implementations of the very same specification.
As such, this fine-tuning is not likely to exist until the codec implementations mature, which likely means that what is available now of HEVC/VP9 serves only as very rough estimates.
Would still interesting though :)
Having done a lot of video encoding myself, I've found that H.264 offers the best quality, especially at lower bit rates.
I really do hope that VP9 can offer the quality of H.265. VP8 just wasn't good enough to compete against H.264.
I really do hope that VP9 can offer the quality of H.265. VP8 just wasn't good enough to compete against H.264.
No, the point is it is good enough, even if in some cases it can't be as good because it has to work around software patents. Theora indeed can't compete in comparison, but VP8 surely can.
Being good enough also depends on the context that the video will be used in. For example, I have an HTML5 video site where closer to 100% of the audience is either FF or Chrome. Both support VP8 out of the box, which means I only have to keep one copy of the video to serve to all users. If I had to target more browsers, such as Safari or IE, I would have to keep two copies of every video in order to work on all browsers (or worse, use flash as a work around).
I am okay with the minor loss in quality, just for the ease of managing a lot of videos and a lot of disk space.
I am okay with the minor loss in quality, just for the ease of managing a lot of videos and a lot of disk space.
Those who really lack support for WebM are mobile users who don't have dedicated hardware decoder and those who use crippled browsers like mobile Safari or mobile IE. Desktop users (at least in case of IE, not sure what's the story with Safari these days) can install needed plugins and DirectShow filters to support WebM. But the lack of dedicated hardware decoding in mobile SoCs remains a serious problem, since even when the browser supports VP8 decoding in software, it would be a degraded experience, especially power consumption wise.
Given how cable looks compared to over the air HDTV, the coming of 4k, and satellite channels even h.264 isn't good enough. VP8 won't be and h.265 is the industries answer. I would expect trouble with the adoption of VP9 given Google's moves with Mortorola's patents.
> I would expect trouble with the adoption of VP9 given Google's moves with Mortorola's patents.
How exactly? There is no indication that Google wants to make VP9 patent encumbered, since their whole idea behind VP8 was to enable high quality open video codec for the Web and beyond. If VP9 will be their natural next step, it will be open as well. If Daala comes soon enough too - it will be another option.
How exactly? There is no indication that Google wants to make VP9 patent encumbered, since their whole idea behind VP8 was to enable high quality open video codec for the Web and beyond. If VP9 will be their natural next step, it will be open as well. If Daala comes soon enough too - it will be another option.
There are patents covering VP8 and I would bet VP9 also is covered.
That's true, and Google explicitly granted their free usage, thus the codec is open in all senses of the word. I'd expect the same story with VP9. So what's your point?
http://www.webmproject.org/license/bitstream/
http://www.webmproject.org/license/software/
http://www.webmproject.org/license/additional/
http://www.webmproject.org/license/bitstream/
http://www.webmproject.org/license/software/
http://www.webmproject.org/license/additional/
Google went back on what Motorola had already pledged when they bought them. What assurance does anyone have that Google won't do the same again?
Motorola was bought from beginning to use as a leverage tool against patent trolls like Apple and MS. What did you expect? VPx is intended to be open from beginning by Google.
Basically, I expect patents licensed a certain way for an important standard to stay licensed that way.
Your post, makes my point. Why would Apple and MS expect Google not to do it again?
Your post, makes my point. Why would Apple and MS expect Google not to do it again?
You can never expect that, because you have no guarantee against unexpected patent trolling. Apple and MS are just looking for excuse to sabotage open codecs. But real threats come from themselves and unknown patent trolls.
The FTC would seem to disagree with you as they believe patent commitments to standards are not changeable after the standard is adopted. Once again Apple and MS don't care if its open, they care about stability.
FTC can't protect against patent trolling. Did you read what's written above?
Google releases VP8 and VP9 with a patent and royalty free licence, so a potential patent threat will hardly come from them.
Motorola released their GSM required patents under FRAND licensing. Google buys Motorola and tries to use those same patents in a fight with Apple and Motorola (dragging Qualcomm in with them). Google has to agree with the FTC not to do that.
Why would Apple or Microsoft believe Google's patent and royalty free license given their behavior with Motorola's prior promises?
Why would Apple or Microsoft believe Google's patent and royalty free license given their behavior with Motorola's prior promises?
Apple and MS just don't like open codecs, period. That's the whole reason they sabotage them. It's not about belief - it's about the struggle between closed and open.
Apple and MS don't like legal uncertainty and additional risks in choosing technologies. Open never entered into it. People also forget some of the problem both companies had with MPEG-LA.
That's bunk. H.264 has as much legal uncertainty as VP8. This didn't stop them from using it. Naturally they just have allergy on open codecs.
H.264 has a group of companies and a lot of lawyers backing it up. VP8 has Google's word. It is not bunk and has nothing to do with open codecs.
> H.264 has a group of companies and a lot of lawyers backing it up.
How can it help against some patent troll who can appear tomorrow and threaten H.264 users? No army of lawyers can help against that. So all this rhetoric is completely useless. Legally VP8 is as good as H.264. The only reason for them to avoid is to retain their monopolistic grip on codecs usage.
How can it help against some patent troll who can appear tomorrow and threaten H.264 users? No army of lawyers can help against that. So all this rhetoric is completely useless. Legally VP8 is as good as H.264. The only reason for them to avoid is to retain their monopolistic grip on codecs usage.
Exactly the patent threat will come from the MPEG-LA or the members directly.
Because if VP9 is indeed similar to H.265 then you I would imagine a patent is being infringed somewhere. And since Google doesn't provide patent indemnification you can guarantee that some big royalties will be demanded from users.
Because if VP9 is indeed similar to H.265 then you I would imagine a patent is being infringed somewhere. And since Google doesn't provide patent indemnification you can guarantee that some big royalties will be demanded from users.
> Exactly the patent threat will come from the MPEG-LA or the members directly. Because if VP9 is indeed similar to H.265
Why would they design the codec to be vulnerable to patent attacks? VP8 was designed to work around threats from H.264. They surely applied the same logic for VP9 since they intend it for practical use and not as theoretical brain exercise. MPEG-LA spits threats all the time, including against Theora and VP8, but they have no teeth to bite.
Why would they design the codec to be vulnerable to patent attacks? VP8 was designed to work around threats from H.264. They surely applied the same logic for VP9 since they intend it for practical use and not as theoretical brain exercise. MPEG-LA spits threats all the time, including against Theora and VP8, but they have no teeth to bite.
I bet you said the same thing back when VP8 was released, no such patent lawsuit has occured.
Also, On2 which Google purchased holds lots of video compression patents, patents which codec's like h.264 and h.265 just as likely violates.
MPEG-LA does not offer any patent indemnification either.
Also, On2 which Google purchased holds lots of video compression patents, patents which codec's like h.264 and h.265 just as likely violates.
MPEG-LA does not offer any patent indemnification either.
Video encoders have a number of tools. It really depends on what tools you have enabled.
The standard itself doesn't specify how an encoder behaves. That leaves a lot of room for innovation on encoders. End of the day, the qp/bitrate ratio you get is encoder implementation dependent and not sometime that's specific to the standard. This is the part where most of the patents exist.
The standard itself doesn't specify how an encoder behaves. That leaves a lot of room for innovation on encoders. End of the day, the qp/bitrate ratio you get is encoder implementation dependent and not sometime that's specific to the standard. This is the part where most of the patents exist.
Considering the sheer number of devices with h.264, it's like saying betamax is superior while everyone only has vhs.
One of the interesting things about VP8 is that its very similar to h.264. Saying that one device has hardware support for h.264 really means that it has support for some of the operations that h.264 uses; likely that same ones that VP8 uses. In fact, I am willing to bet that is why Android devices 2.3.3 and up require VP8 support; it was easy to add it in.
The ffmpeg guys wrote a VP8 decoder with 1400 lines of C because of how much they were able to reuse. From: http://blogs.gnome.org/rbultje/2010/06/27/googles-vp8-video-...
The ffmpeg guys wrote a VP8 decoder with 1400 lines of C because of how much they were able to reuse. From: http://blogs.gnome.org/rbultje/2010/06/27/googles-vp8-video-...
I don't quite understand, Betamax had superior quality, but in this case this article shows that h264 has more detail at the same bit rate combined with much faster encoding.
Does your standalone device have vp8? All bluray devices can play h264 streams but almost never vp8.
Of course as everything switches to android this will change completely by the end of the decade, but for streaming in 2013, h264 is the better investment.
Once they move to h265 vs vp9 the point is moot but that won't be common for at least five years. The encoding requirements for those streams will also be significant.
Of course as everything switches to android this will change completely by the end of the decade, but for streaming in 2013, h264 is the better investment.
Once they move to h265 vs vp9 the point is moot but that won't be common for at least five years. The encoding requirements for those streams will also be significant.
SoC manufacturers slowly add VP8 support. But it's limited to the newest models at most. So it's coming but not as rapidly as you could wish.
> Of course as everything switches to android this will change completely by the end of the decade
That's hilarious.
That's hilarious.
You disagree that a free, solid OS like android with a commercial marketplace like the google play store will become ubiquitous?
I mean we are reaching the point where a $20 device can just plug right into a hdmi port and you have a full blown pc with standardized wifi and bluetooth support. By 2018 they will be $10 and sold at gas stations.
When that happens you can have any codec upgrade you want as long as it can still decode on that hardware spec in realtime.
I mean we are reaching the point where a $20 device can just plug right into a hdmi port and you have a full blown pc with standardized wifi and bluetooth support. By 2018 they will be $10 and sold at gas stations.
When that happens you can have any codec upgrade you want as long as it can still decode on that hardware spec in realtime.
Only if you limit your discussion to mobile platforms. Most recent computers should be able to run VP8 video without the end user taking notice (assuming a generic player like VLC is installed), even if they have to do software decoding.
"While I'd love a patent- and royalty-free video format to offer the highest quality compression you can find, I wouldn't hold my breath for one."
I wonder why couldn't they allow software usage ( Encoder and Decoder ) to be free. And only collect royalty on Hardware, and Industry Production use.
I wonder why couldn't they allow software usage ( Encoder and Decoder ) to be free. And only collect royalty on Hardware, and Industry Production use.
Oh look another article that's been flagged into oblivion by zealots.
{currently at the bottom of page 4 when by pts it should still be in the top 10)
{currently at the bottom of page 4 when by pts it should still be in the top 10)