WebM and WebP Hand Ported to JavaScript for All Browsers(badassjs.com)
badassjs.com
WebM and WebP Hand Ported to JavaScript for All Browsers
http://badassjs.com/post/17218459521/webm-and-webp-hand-ported-to-javascript-for-all
19 comments
I'm not sure this is a practical approach for "real" video playback.
Letting the browser abstract away video playback, doing whatever's optimal on the target hardware (hardware acceleration, optimized SSE/NEON vector instructions, and so on) will always be better than decoding in software, and the functionality already exists for us via the <video> tag - decoding via JavaScript is in many ways a big step backwards.
I understand that a JavaScript VP8 decoder provides greater cross-browser compatibility, but personally I think it's worth the overhead/cost of encoding and storing both H.264 and VP8/WebM versions of videos in order to win sane mobile playback and reduced CPU usage on the desktop (via hardware H.264 acceleration).
Plus, this decoder doesn't consider audio, as far as I can tell. HTML5 provides very bad support for programmatically-generated audio with strict time constraints - this is a common problem with writing HTML5-based games as well and I can't see any way that any browser today could produce adequately synchronized, real-time audio along with video rendered to a canvas.
This is an absolutely awesome piece of code, and the hacker in me loves it, but I don't think it's practical in real life. It might have limited use in audio-less product demos, page banners, and the like, but for legitimate video playback, I think the <video> tag combined with a flash fallback is vastly superior, even when the need to encode the desired video to both H.264 and WebM is taken into account.
Letting the browser abstract away video playback, doing whatever's optimal on the target hardware (hardware acceleration, optimized SSE/NEON vector instructions, and so on) will always be better than decoding in software, and the functionality already exists for us via the <video> tag - decoding via JavaScript is in many ways a big step backwards.
I understand that a JavaScript VP8 decoder provides greater cross-browser compatibility, but personally I think it's worth the overhead/cost of encoding and storing both H.264 and VP8/WebM versions of videos in order to win sane mobile playback and reduced CPU usage on the desktop (via hardware H.264 acceleration).
Plus, this decoder doesn't consider audio, as far as I can tell. HTML5 provides very bad support for programmatically-generated audio with strict time constraints - this is a common problem with writing HTML5-based games as well and I can't see any way that any browser today could produce adequately synchronized, real-time audio along with video rendered to a canvas.
This is an absolutely awesome piece of code, and the hacker in me loves it, but I don't think it's practical in real life. It might have limited use in audio-less product demos, page banners, and the like, but for legitimate video playback, I think the <video> tag combined with a flash fallback is vastly superior, even when the need to encode the desired video to both H.264 and WebM is taken into account.
Web APIs for audio are under active development. The W3C has formed a working group where use cases and two specifications (Web Audio API and MediaStream Processing API) are being developed. Chrome and Safari are shipping experimental versions of the Web Audio API now, and Firefox will soon be shipping experimental versions of the MediaStream Processing API (it's available now in a special build). Firefox 4 onwards also includes the simpler Audio Data API. I believe all of these APIs provide sufficient functionality to build the audio portions of media playback with working audio-visual synchronization.
Good to hear - my last experience with Web Audio API left something to be desired, but the MediaStream Processing API seems ideal for this use case.
I think my points about hardware acceleration and the <video> tag still stand, but it sounds like I was too pessimistic.
I think my points about hardware acceleration and the <video> tag still stand, but it sounds like I was too pessimistic.
I would be more concerned about the memory consumption (especially on mobile).
On ios in particular I recall[1] something about jpg taking up significantly more than their file size in memory, due to decoding the image. png's don't suffer from that (an ios optimization for pngs or something), but they tend to be overall larger for big files (retina size).
I would think webp.js would be even worse off than jpg in regard to memory usage for mobile devices. (note: this is just a guess/hunch though. I saw no data one way or the other on the linked post, nor on the libwebp site).
[1]: Maybe a fellow HN'er knows more about this particularity with regard to ios image memory consumption.
edit: found this: http://cubiq.org/testing-memory-usage-on-mobile-safari which appears to rebut the jpg/png memory discrepancy. Perhaps our particular app was doing something odd with the jpg images we were testing, that was causing the additional memory usage. Maybe it had to do with loading in the actual jpg decoding code itself (png perhaps already loaded?).
edit2: this article seems to confirm it: http://mobile.tutsplus.com/tutorials/mobile-design-tutorials...
subtext: test test test! :P
On ios in particular I recall[1] something about jpg taking up significantly more than their file size in memory, due to decoding the image. png's don't suffer from that (an ios optimization for pngs or something), but they tend to be overall larger for big files (retina size).
I would think webp.js would be even worse off than jpg in regard to memory usage for mobile devices. (note: this is just a guess/hunch though. I saw no data one way or the other on the linked post, nor on the libwebp site).
[1]: Maybe a fellow HN'er knows more about this particularity with regard to ios image memory consumption.
edit: found this: http://cubiq.org/testing-memory-usage-on-mobile-safari which appears to rebut the jpg/png memory discrepancy. Perhaps our particular app was doing something odd with the jpg images we were testing, that was causing the additional memory usage. Maybe it had to do with loading in the actual jpg decoding code itself (png perhaps already loaded?).
edit2: this article seems to confirm it: http://mobile.tutsplus.com/tutorials/mobile-design-tutorials...
subtext: test test test! :P
I believe the point of WebP is that it is smaller than JPEG and PNG files. And the JS file is only 27 KB minified and gzipped...
Google claims an average-case 25-34% decrease in file sizes from JPEG and PNG, so I guess if you have more than ~100K worth of images to serve it would be worth your while, bandwidth-wise (though of course rendering time on the client would go way up).
Rendering memory usage could change, too (most likely upwards). I haven't looked at the source, but I guess this renders to canvas and keeps the uncompressed data around in that canvas. Browsers might opt to uncompress images on the fly when needed.
Your calculation assumes that no browser natively supports WebP but you'd only need to send the .js and rendering time would only go up (noticeably) if the browser didn't support WebP natively so if your audience was 99% Chrome (& Opera & Android4+ & mobile apps that can include the C library etc.) then you'd be much more likely to use this fallback for the other 1% than than if it was 99% IE.
For the current WebP support figures globally it's about 3/10ths and Firefox support could double that to 6/10ths. I wonder if it already makes sense for Wikipedia? I guess it comes down to the cost of double storage/transcode vs bandwidth.
For the current WebP support figures globally it's about 3/10ths and Firefox support could double that to 6/10ths. I wonder if it already makes sense for Wikipedia? I guess it comes down to the cost of double storage/transcode vs bandwidth.
27K is pretty big in the compressed image world - plus, you have to consider the amount of time the JavaScript requires to render WebP (which could instead have been used to transfer a few extra KB of data).
I think it's pretty hard to win this way.
I think it's pretty hard to win this way.
Do I understand it correctly that the whole codec runs in javascript?
I have mixed feelings about handling web media like this. It's probably very well-done, interesting javascript, but I don't know if this is a long-term solution for the web. I feel like high-performance decoding should be done in the browser and is not Javascript's domain.
But many people want to use free video formats on the web and Apple and Microsoft are still refusing to ship them.
This seems like the right approach going forward, as long as JavaScript engines continue to approach the performance of native code. This approach keeps codec implementations in the JavaScript sandbox where they can't provide new exploits, rather than extending the attack surface with large, frequently exploited codecs.
In general, JavaScript seems like the right approach for anything that doesn't require special privileges to do, and for that the APIs should focus on creating the minimum possible interface that lets JavaScript do all the interesting work.
In general, JavaScript seems like the right approach for anything that doesn't require special privileges to do, and for that the APIs should focus on creating the minimum possible interface that lets JavaScript do all the interesting work.
I'd like to agree, but this makes a good benchmark, and right now it's showing that JavaScript is about 20 times slower than native code on my machine in both Chrome and Firefox. That's completely unacceptable for video decoding. In fact, even if JavaScript was just as fast as native code it would still be unacceptable for mobile video, where hardware acceleration is required for acceptable performance and battery life.
Just wait until browsers start supporting WebCL and we can do video decoding on the graphics card :)
I'm betting WebCL running on CPU cores will beat JS by 20x margin as well.
How's WebP compared to ImageZero?
https://kdepepo.wordpress.com/2012/01/30/fast-lossless-color...
https://kdepepo.wordpress.com/2012/01/30/fast-lossless-color...
WebP is, by default, lossy whereas ImageZero is lossless so they're not directly comparable. WebP does have an in-progress lossless mode though, known as WebPll.
WebPll compresses better, but takes much longer to do it, therefore is better suited to website images which are encoded once and transmitted multiple times.
ImageZero compresses fast, but not as well, therefore is better suited to saving images you are working on in a image editor to disk.
WebPll compresses better, but takes much longer to do it, therefore is better suited to website images which are encoded once and transmitted multiple times.
ImageZero compresses fast, but not as well, therefore is better suited to saving images you are working on in a image editor to disk.
I wonder if the file size gains of switching to WebP is worth the hit of including the WebP.js file. Of course you'd now require clients to support JS but in this day and age that's reasonable anyways.