Web Audio API comes to Firefox(hacks.mozilla.org)
hacks.mozilla.org
Web Audio API comes to Firefox
https://hacks.mozilla.org/2013/07/web-audio-api-comes-to-firefox/
14 comments
Great to hear that! Is this project publicly available somewhere by any chance?
Unfortunately it's not at the moment.
This is great news. I've been playing around with audio streams generated by script (DAC emulation) on Chrome/Safari and now I can try Firefox.
A key problem (and measure of quality) for me is reducing the latency. I've had trouble getting continuous sound with less than a quarter second of buffer. (Ideally, I'd like to achieve smooth sound with only 17ms of buffer/lag). Another browser to work with will help, and, maybe the IE team will take notice.
A key problem (and measure of quality) for me is reducing the latency. I've had trouble getting continuous sound with less than a quarter second of buffer. (Ideally, I'd like to achieve smooth sound with only 17ms of buffer/lag). Another browser to work with will help, and, maybe the IE team will take notice.
If you hit latency issues in Firefox, please get in touch with me. Note that we currently have a rather high latency on Android, which we have plans to fix, but things should work fairly well on desktop platforms.
Fantastic to see some of these things come to reality. In our real-time video entertainment game we've had to code all sorts of tricks in the client to get sound events to occur in sync across clients.
Timed events is the most important aspect of creating a game or entertainment experience that sounds real to users. I completely took it for granted before I started on this project. Time is quite a rubbery material right now in browsers!
Timed events is the most important aspect of creating a game or entertainment experience that sounds real to users. I completely took it for granted before I started on this project. Time is quite a rubbery material right now in browsers!
Fantastic news. I've used Web Audio under Chrome for a couple of experimental projects now, and it really is a fun API to work with.
For those interested, Apple documents the Web Audio API pretty well: http://developer.apple.com/library/safari/#documentation/Aud...
I have made something a little more advanced with Web Audio: http://mixbolt.com
Cant wait to have support on all the browsers (especially mobile) :)
Good stuff!
Our mobile & desktop web app, SpeakerBlast, uses the web audio API to play audio in sync across multiple devices.
Though up until now it only works in Chrome & in mobile Safari.
Our mobile & desktop web app, SpeakerBlast, uses the web audio API to play audio in sync across multiple devices.
Though up until now it only works in Chrome & in mobile Safari.
If you hit problems porting that to Firefox, please get in touch with me, would be happy to help you!
That's huge! I've been waiting for this for quite some time.
If Firefox also got some more UI polish it would be my fave browser.
This is great news! Been waiting for this. Now if only I don't need to use javascript for this. :)
Excellent. Although Firefox previously had a simple API that could be used to do most of these things manually in JavaScript, the Web Audio API provides convenient high level functionality while retaining the ability to manually process samples if needed, and, of course, now works in both WebKit and Firefox.
For whatever it's worth, we have deprecated support for the Mozilla Audio Data API in favor of Web Audio, and will remove support for it in the near future.
Good show, but I wonder how far we can really push the web browser, how will this affect desktop/native apps? We have already canvas and webgl and all but I dont see a web photoshop version competing with the real deal, how come? We're here giving access to "web-scripts" to more and more OS functions and closer to hardware, and yet the most productive apps are native, hm?
I'm not sure that the point is necessarily that the browser should be competing with native pro apps.
Web apps tend to be really good at very specialized functions. A silly, but I think still valid example would be a meme generator using <canvas> and JS. If you want a picture of a cat with some white Arial text and a black drop shadow, a web app is going to be great. It'll probably be faster than Photoshop (especially if you count startup time), and way more accessible.
Same thing goes for the Web Audio API. Will we see a full-fledged Pro Tools competitor? Probably not. But we will see ringtone generators and voice recorders and (hopefully!) some T-Pain AutoTune apps.
Great web apps are all about playing off the web's strengths, not trying to replicate the desktop.
Web apps tend to be really good at very specialized functions. A silly, but I think still valid example would be a meme generator using <canvas> and JS. If you want a picture of a cat with some white Arial text and a black drop shadow, a web app is going to be great. It'll probably be faster than Photoshop (especially if you count startup time), and way more accessible.
Same thing goes for the Web Audio API. Will we see a full-fledged Pro Tools competitor? Probably not. But we will see ringtone generators and voice recorders and (hopefully!) some T-Pain AutoTune apps.
Great web apps are all about playing off the web's strengths, not trying to replicate the desktop.
How man years has it taken to get photoshop where it is today? Javascript and internet speeds have only recently gotten fast enough to compete. Give it another 5-10 years and then we'll see real competition for large desktop apps.
Does this mean Firefox will be able to play MP3s now?
We currently support MP3 playback for Windows Vista+ and Android if I'm not mistaken, and support for other platforms is being worked on. Web Audio will support all of the codecs supported by <audio> depending on the platform.
Thanks! MP3 support for OSX would be great. I was recently working on a music playing web app and was frustrated by this chart http://www.w3schools.com/html/html5_audio.asp
You can follow this bug https://bugzilla.mozilla.org/show_bug.cgi?id=851290 if you're interested. :)
And on Linux, if you enable gstreamer and if your gstream has mp3 support installed. (A lot of ifs, but it is the Linux desktop ...)
Yes, that should be available in Nightly and Aurora:
https://bugzilla.mozilla.org/show_bug.cgi?id=794282
https://bugzilla.mozilla.org/show_bug.cgi?id=794282
After trying to implement it, I've grown frustrated by the Web Audio spec. It has a lot of nice features, but they're ultimately justified by JS being slow. The spec itself is not very tight considering how much it covers - for example, I can easily see parameter changes behaving differently across different implementations. The real tests of the API haven't appeared yet, and if they fail all those features will be a waste because nobody will want to ship on it.
Do you have specific complaints? We're definitely eager to hear about them at the Audio working group.
There is no specification for the computation of filter coefficients.
Audio Nodes are subject to garbage collection delays. This effectively makes the API no-go for live performance situations. (I know you can't fix it either.)
Parameter change specifications contain either an analytic or a mutable form, but in practical implementations, both forms become desirable to implement efficient processing that can also begin at arbitrary times within the envelope. I cancelled my implementation of the API after discovering that the lack of detail in this section made for a fragile, confusing, trial-and-error situation to prevent discrepancies when comparing the results of the analytic form against the mutable one.
Rendering does not appear to have any automatic regression test available. Web Audio needs an Acid3-style test.
Audio Nodes are subject to garbage collection delays. This effectively makes the API no-go for live performance situations. (I know you can't fix it either.)
Parameter change specifications contain either an analytic or a mutable form, but in practical implementations, both forms become desirable to implement efficient processing that can also begin at arbitrary times within the envelope. I cancelled my implementation of the API after discovering that the lack of detail in this section made for a fragile, confusing, trial-and-error situation to prevent discrepancies when comparing the results of the analytic form against the mutable one.
Rendering does not appear to have any automatic regression test available. Web Audio needs an Acid3-style test.
About the first and third points, it would be great if you can send detailed feedback to this mailinglist: http://lists.w3.org/Archives/Public/public-audio/. Doing that would give us a chance to discuss these issues and fix the spec if possible.
About the second point, that's not accurate. All audio processing in a conforming Web Audio implementation happens on a background thread reserved for audio processing, which does not need to wait for things happening on the main thread (including GC pauses.) In practice, Gecko's processing model uses message passing between the audio thread and the main Javascript thread. WebKit and Blink use locks to synchronize the two threads some of the times, but the locking is minimal and it should not have a huge impact on performance.
About the testing issue, the working group has started to work on it recently, and you can see the beginning of the work here: https://github.com/w3c/web-platform-tests/tree/master/webaud.... We're planning to integrate the regression tests from both Mozilla and WebKit into the test suite for the spec (http://mxr.mozilla.org/mozilla-central/source/content/media/... and http://trac.webkit.org/browser/trunk/LayoutTests/webaudio) into the test suite. And of course, contributions for more tests are always welcome.
About the second point, that's not accurate. All audio processing in a conforming Web Audio implementation happens on a background thread reserved for audio processing, which does not need to wait for things happening on the main thread (including GC pauses.) In practice, Gecko's processing model uses message passing between the audio thread and the main Javascript thread. WebKit and Blink use locks to synchronize the two threads some of the times, but the locking is minimal and it should not have a huge impact on performance.
About the testing issue, the working group has started to work on it recently, and you can see the beginning of the work here: https://github.com/w3c/web-platform-tests/tree/master/webaud.... We're planning to integrate the regression tests from both Mozilla and WebKit into the test suite for the spec (http://mxr.mozilla.org/mozilla-central/source/content/media/... and http://trac.webkit.org/browser/trunk/LayoutTests/webaudio) into the test suite. And of course, contributions for more tests are always welcome.
I'm not sure how reverting to a lower level JS-driven API could fix the performance issues. High performance audio, like 2D and 3D graphics, needs the ability to offload to native code or dedicated hardware. Writing a DSP in JS makes things portable, but so does writing an OpenGL rasterizer in CPU code. OpenGL to this day still has portability issues between GPUs, but the benefits outweigh the disadvantages.
The only solution is IMHO is to keep finding holes in the spec, driven by feedback from application developers, and plugging the holes.
I don't think a spec/standards committee can fix all of these issues, and like with OpenGL, I suspect, we'll need Web Audio 1.1/1.2/1.3/etc
The only solution is IMHO is to keep finding holes in the spec, driven by feedback from application developers, and plugging the holes.
I don't think a spec/standards committee can fix all of these issues, and like with OpenGL, I suspect, we'll need Web Audio 1.1/1.2/1.3/etc
Absolutely. Feedback from developers testing their applications and ideas with the existing implementations is crucial to having a high quality spec, so if you have specific issues, please provide feedback here: http://lists.w3.org/Archives/Public/public-audio/. We'll definitely take that into consideration and we can always amend the spec based on the feedback we receive to improve things.
Echoing Ehsan's comments, please, if you have specific feedback on the spec we'd really appreciate to hear your views in the W3C Audio Working Group[1].
We're approaching the stage where we'd like to go to "last call", so there's never been a better time to give us some feedback. Your experience of trying to implement the spec would be very valuable to us.
If you have any questions about getting involved in the group please contact me directly (I co-chair the WG) and I'll help you.
[1] http://lists.w3.org/Archives/Public/public-audio/
We're approaching the stage where we'd like to go to "last call", so there's never been a better time to give us some feedback. Your experience of trying to implement the spec would be very valuable to us.
If you have any questions about getting involved in the group please contact me directly (I co-chair the WG) and I'll help you.
[1] http://lists.w3.org/Archives/Public/public-audio/
I built a fairly basic ambient drone generator with the Chrome implementation of the Web Audio API:
http://matt-diamond.com/drone.html
Haven't gotten a chance to add Firefox compatibility yet (working on it!).
http://matt-diamond.com/drone.html
Haven't gotten a chance to add Firefox compatibility yet (working on it!).
That sounds great! Please get in touch with me if you hit any problems, would be happy to help!
if (typeof AudioContext !== "undefined") { this.context = new AudioContext(); }
It worked without modification, and the audio sounds great so far. In the meantime I'm still falling back to webkitAudioContext for Chrome and Safari until they catch up.