Why HTML5 Media is not Enough(ofmlabs.org)
ofmlabs.org
Why HTML5 Media is not Enough
http://ofmlabs.org/articles/dublin.html
1 comments
Of course, at ofmlabs we all want sink.js to be made unnecessary as soon as possible! It's just a pragmatic solution when you want to make an audio demo and cannot wait 3 years for the browser implementors to reconcile :)
In the long term, we have been participating in the W3C Audio WG, and discussing with various folks at Google and Mozilla, to try and understand where the debate is going. Robert O'Callahan's MediaStream API seems the most promising so far.
We are not advocating the use of sink.js as a long term solution!
In the long term, we have been participating in the W3C Audio WG, and discussing with various folks at Google and Mozilla, to try and understand where the debate is going. Robert O'Callahan's MediaStream API seems the most promising so far.
We are not advocating the use of sink.js as a long term solution!
I have no criticisms for sink.js my comment was more on the philosophical side. The danger in a solution like sink.js is it becomes "good enough" until people need more than it can provide. Then they turn to the APIs and end up dealing with fragmentation.
If you think about it that's exactly what happened with IE circa Version 5. The one good thing IE did was make it possible to develop for one browser. What the industry should have done then was what they're doing with HTML5 now. Define a standard so other competitors could enter the market without creating fragmentation. But that's not what happened and now everyone has to write for 3 browsers (if not more thanks to IE7,8,and 9)
Same with sink.js. For all the bad in Flash almost everyone has it and it's a consistent thing you can write to. If people use sink.js now and don't pay attention to the fragmented APIs underneath we'll end up having to write to 3 different APIs down the line.
So again I'm not against sink.js. I'm just saying people using it need to realize exactly what you said and pressure browsers for a common API in the long run.
If you think about it that's exactly what happened with IE circa Version 5. The one good thing IE did was make it possible to develop for one browser. What the industry should have done then was what they're doing with HTML5 now. Define a standard so other competitors could enter the market without creating fragmentation. But that's not what happened and now everyone has to write for 3 browsers (if not more thanks to IE7,8,and 9)
Same with sink.js. For all the bad in Flash almost everyone has it and it's a consistent thing you can write to. If people use sink.js now and don't pay attention to the fragmented APIs underneath we'll end up having to write to 3 different APIs down the line.
So again I'm not against sink.js. I'm just saying people using it need to realize exactly what you said and pressure browsers for a common API in the long run.
You're right about IE5 (then 6) stagnating, but the predicate there was Microsoft's OS monopoly and browser tying. No such condition today -- we have a much more competitive browser market, and no browser can afford to self-stagnate.
Roc says he is nearly done with MediaStreams and if that looks good to enough people on the standards bodies, or really among the browser implementors, then it'll go forward.
Roc says he is nearly done with MediaStreams and if that looks good to enough people on the standards bodies, or really among the browser implementors, then it'll go forward.
> But isn't that exactly what Flash is?
JS and the various browser APIs for media come with the browser. Flash doesn't necessarily. You can count on Chrome and Firefox having a media API, but not Flash.
JS and the various browser APIs for media come with the browser. Flash doesn't necessarily. You can count on Chrome and Firefox having a media API, but not Flash.
At this point, can I count on a user having Flash over Chrome or Firefox? Today, I'd say that a user having Flash is a more reliable bet than them having Chrome/Firefox. And if they have Chrome, they have Flash.
I get where you're going, but this conversation always gets dicey. People want Flash to die, so they sit around the campfire dreaming about why it is bad, but fail to see why it has succeeded thus far.
If we really want Flash to die, we should be concentrating on advancing the standard (or abandon the slow-moving committees involved).
Instead, this rhetoric of how Flash performs, or its inherit proprietary format gets brought up. Over and over. Ad nauseum.
And yet Flash – today – outperforms the alternatives in almost every realm of multimedia. Audio API, Camera support, 3D/vector drawing, video decoding, licensing, streaming.
Drama and vilification won't change the fact that I still can't allow a user to upload a video with a webcam using standard web technologies – probably not until 2020 (or whatever other gratuitously pushed off date it may be). I still can't reliably build a highly interactive, full-width experience without the browser having a fit.
This is the reality.
I get where you're going, but this conversation always gets dicey. People want Flash to die, so they sit around the campfire dreaming about why it is bad, but fail to see why it has succeeded thus far.
If we really want Flash to die, we should be concentrating on advancing the standard (or abandon the slow-moving committees involved).
Instead, this rhetoric of how Flash performs, or its inherit proprietary format gets brought up. Over and over. Ad nauseum.
And yet Flash – today – outperforms the alternatives in almost every realm of multimedia. Audio API, Camera support, 3D/vector drawing, video decoding, licensing, streaming.
Drama and vilification won't change the fact that I still can't allow a user to upload a video with a webcam using standard web technologies – probably not until 2020 (or whatever other gratuitously pushed off date it may be). I still can't reliably build a highly interactive, full-width experience without the browser having a fit.
This is the reality.
Flash does not outperform every aspect of the alternatives. The standard flash browser plugin can't even provide frame accurate playback, something that can be done with the video tag in chrome, IE9, Safari and Firefox.
It does not out perform chrome's webgl, audio API, video decoding support, except in the cross platform support it offers.
The only benefit it has is that it Is common, but there can be version differences on feature support.
It does not out perform chrome's webgl, audio API, video decoding support, except in the cross platform support it offers.
The only benefit it has is that it Is common, but there can be version differences on feature support.
Reality is also that Flash won't be supported in the browsers of mobile devices.
The reason that we complain about performance, and the proprietary format, is just because that are the main problems with it.
Building without flash is not for everyone, but if you want to support Mobile Safari, or for all the other people without access to flash, you need a javascript solution to the problem.
Have faith, even IE will be able to support rich multimedia applications in the future, or it will become irrelevant.
That future is starting to become available now in Firefox, Safari and Chrome.
The reason that we complain about performance, and the proprietary format, is just because that are the main problems with it.
Building without flash is not for everyone, but if you want to support Mobile Safari, or for all the other people without access to flash, you need a javascript solution to the problem.
Have faith, even IE will be able to support rich multimedia applications in the future, or it will become irrelevant.
That future is starting to become available now in Firefox, Safari and Chrome.
The debate isn't about whether Flash is more capable than HTML5 - more whether HTML5 is a better 'model' than Flash and when I say better I mean that 1. It is not owned by any one company and 2. You can view the source.
Viewing the source has been fundamental to the evolution of the web and it will be fundamental going forward - with Flash you can't do it. Not to mention that Flash's sand-boxed implementation doesn't play nicely with other HTML elements. (Try manipulating Flash video with Canvas).
But what the author is really talking about is changing the process of establishing standards - bottom up rather than top down. Give us, the developers the basic building blocks and we will build the rest. It should become obvious then what to implement as a standard, if anything.
I recently read Paul Graham's Hackers and Painters and although I certainly don't agree with all he says, one paragraph stood out:
"Let yourself be second-guessed. When you make any tool, people use it in ways you didn't intend, and this is especially true of a highly articulated tool like a programming language. Many a hacker will want to tweak your semantic model in a way that you never imagined. I say let them. Give the programmer access to as much internal stuff as you can."
Viewing the source has been fundamental to the evolution of the web and it will be fundamental going forward - with Flash you can't do it. Not to mention that Flash's sand-boxed implementation doesn't play nicely with other HTML elements. (Try manipulating Flash video with Canvas).
But what the author is really talking about is changing the process of establishing standards - bottom up rather than top down. Give us, the developers the basic building blocks and we will build the rest. It should become obvious then what to implement as a standard, if anything.
I recently read Paul Graham's Hackers and Painters and although I certainly don't agree with all he says, one paragraph stood out:
"Let yourself be second-guessed. When you make any tool, people use it in ways you didn't intend, and this is especially true of a highly articulated tool like a programming language. Many a hacker will want to tweak your semantic model in a way that you never imagined. I say let them. Give the programmer access to as much internal stuff as you can."
Adobe sits and thinks about what the next steps are for Flash and web standards organizations sit and think about how to replicate what Flash does – understand the demand cycle.
My point isn't that HTML5 sucks, it's that people see it as better than Flash – but it isn't, sadly. That isn't an evangelistic statement, it is a factual one. To push forward, we can't be ignorant. We (as a community) have to look at what Flash does and take it for HTML5 and then come back to the table at least as often as Adobe. Otherwise, Flash will always be better.
My point isn't that HTML5 sucks, it's that people see it as better than Flash – but it isn't, sadly. That isn't an evangelistic statement, it is a factual one. To push forward, we can't be ignorant. We (as a community) have to look at what Flash does and take it for HTML5 and then come back to the table at least as often as Adobe. Otherwise, Flash will always be better.
The point is both Adobe and the W3C are guessing. Think how much faster we could advance if it was put in the hands of the community. A community that constantly implements, gets feedback, re-implements, collaborates. You only have to look at three.js (https://github.com/mrdoob/three.js/) and similar to get an idea of what can be built on a low-level API.
[Edit: guessing, not second-guessing]
[Edit: guessing, not second-guessing]
Having participated in web browser standards for 15 years, in good (competitive) markets and bad, I agree completely, without reservation even though perfection is not an option.
The developer community, especially with open source as practiced on github.com, is much better able to path-find better high-level models and APIs. Committees and individual browser vendors are less likely to find the right designs and get them codified as well or as quickly.
This is not inevitable. You could have a righteous hacker/designer at a browser company, whose API and implementation are so winning they sweep all before them. Great if this happens, but it's rare in my experience.
Thus the apparent paradox of low-level APIs and increasingly very fast JS engines enabling faster and better hacker-community-based de-facto standardization than even the modern browser vendors can achieve on average. Then the standards bodies ideally roll up de-jure standards based on uncontested winning designs.
We're in the midst of this, so it's hard to see it in full. It's also slower than some people want, but Flash was not built in a day, or a year, either.
Browser vendor "defection" (failure to cooperate in standards bodies on consensus standardization) is an ongoing threat. Competition as browsers merge with mobile and desktop/tablet OS front-sides (home screens) and app platforms is of course still required to keep standards bodies functional.
The developer community, especially with open source as practiced on github.com, is much better able to path-find better high-level models and APIs. Committees and individual browser vendors are less likely to find the right designs and get them codified as well or as quickly.
This is not inevitable. You could have a righteous hacker/designer at a browser company, whose API and implementation are so winning they sweep all before them. Great if this happens, but it's rare in my experience.
Thus the apparent paradox of low-level APIs and increasingly very fast JS engines enabling faster and better hacker-community-based de-facto standardization than even the modern browser vendors can achieve on average. Then the standards bodies ideally roll up de-jure standards based on uncontested winning designs.
We're in the midst of this, so it's hard to see it in full. It's also slower than some people want, but Flash was not built in a day, or a year, either.
Browser vendor "defection" (failure to cooperate in standards bodies on consensus standardization) is an ongoing threat. Competition as browsers merge with mobile and desktop/tablet OS front-sides (home screens) and app platforms is of course still required to keep standards bodies functional.
Just a quick reply to your "I still can't allow a user to upload a video with a webcam using standard web technologies" point. I believe you can or soon will be able to upload video captured on a mobile device in Firefox for Android and the Android stock browser, using
<input type="file" device="camera" accept="video/mpeg"/>
"To make it work in both Chrome and Firefox, we had to use sink.js, to abstract the differences between the audio APIs."
But isn't that exactly what Flash is? Yes sink.js would be an open solution but it would be an open solution based on proprietary APIs built into the browser (which themselves are open source but which can't be changed without approval of their governing bodies making them not that much different than Flash and its product teams)
So really we've traded Flash's native audio processing for a JavaScript solution using a Web API which in turn uses native APIs.
It seems like it would be more productive to force the browsers to develop one common API