Mozilla Demos MediaStream Processing, Audio Mixing in Firefox
Mozilla is drafting a proposal for a new web standard called MediaStream Processing that introduces JavaScript APIs for manipulating audio and video streams in real time. The specification is still at an early stage of development, but Mozilla has already started working on an implementation for testing purposes.
Mozilla’s Robert O’Callahan, the author of the MediaStream Processing API proposal draft, released experimental Firefox builds that include MediaStream Processing support. He has also published a set of demos (note: you need to run the experimental build to see the demos) that illustrate some of the functionality defined by the specification.
The demos show how the APIs can be used to perform tasks like rendering a visualization of a video’s audio track in a Canvas element while the video is playing. It also shows how the APIs can be used for mixing tasks, like implementing a cross-fade between two audio streams, dynamically adjusting the volume of a video, and programmatically generating audio streams.
One of the characteristics that distinguishes the MediaStream Processing API from previous web audio API proposals is that it aims to interoperate better with existing web standards. For example, it relies on the MediaStream interface in the WebRTC specification. It also allows users to take advantage of Web Workers for threading and will work with getUserMedia to eventually support real-time manipulation of streams from microphones and webcams.
The current implementation of the specification focuses on audio capabilities. As O’Callahan explained this week in a blog post, support for video manipulation will be added in the future when the necessary graphics APIs are accessible through Web Workers. MediaStream Processing on video will be useful for doing things like QR code recognition and augmented reality in web applications, he said.
Analyzing a video's audio track and visualizing it in real time
So, when will this functionality be available in a stable Firefox release? It might take some time. According to O’Callahan, the patch needs some cleanup before the functionality can land in trunk and make it into regular nightly builds. Even then, the MediaStream Processing functionality likely won’t be generally available until the specification has solidified.
“The biggest limitation is that it’s not shipping in Firefox yet. My giant patch is messy and a lot of cleanup needs to be done. I have a plan to split the patch up, clean up the pieces and land them piecemeal. In particular I need to get some of the infrastructure landed ASAP to help the WebRTC team make progress,” he wrote. “When we ship it, much or all of the API will probably be disabled by default, behind a hidden pref, until the standards situation is resolved.”
MediaStream Processing is definitely going to be worth the wait. Some fantastic capabilities are going to be unlocked when the specification is fully implemented. It will open the door for using native web standards to perform some sophisticated real-time media processing tasks that were previously only possible with browser plugins.
This article originally appeared on Ars Technica, Wired’s sister site for in-depth technology news.
CSS1K Showcases How Developers Can Do More With Less
CSS1K: light on the file size, heavy on variety
When the web standards movement first began to take off years ago one of its primary advocates, Dave Shea, launched CSS Zen Garden to showcase what designers could do using only web standards. CSS Zen Garden proved a huge hit, drawing hundreds of contributions from web developers around the world.
The web has changed considerably since the heyday of CSS Zen Garden. Today we can do just about anything using web standards like HTML and CSS, but now there’s a new problem — websites are getting fatter, as much as 25 percent fatter in the last year alone. Much of that added weight comes from JavaScript, but CSS shares some of the blame. According to HTTP Archive, the top 1,000 sites on the web load an average 30KB worth of CSS.
Inspired by CSS Zen Garden and driven by a desire to showcase what can be done with a mere 1KB of CSS, Jacob Rask, a web developer at Opera Software, created CSS1K about six months ago. Since then the site has showcased 68 different designs, all with stylesheets under 1KB.
The site is modeled on CSS Zen Garden, with a fixed set of HTML tags and content. Using that HTML base designers can then bend the basic frame to whatever look they’ve come up with — provided the total minified size of the CSS file is only 1KB.
Call it an effort in minimalism if you like, or simply think of it as a design constraint; either way designs submitted to CSS1K demonstrate just how much is possible with very little CSS. While no one is suggesting every website needs to be so constrained, playing with CSS1K is a nice reminder that you don’t always need huge stylesheets to create a nice-looking website.
If you’d like to participate in the CSS1K fun just fork CSS1K at GitHub and send your submission as a pull request. There is essentially only one rule — you can’t use data URIs, @font-face or @import statements to skirt the 1K CSS filesize limit. Note, however, that your 1K CSS file can be minified and any browser prefixes you need don’t count toward the total (in fact, you submit your code unprefixed and CSS1K adds the prefixes for you). For the full details and file requirements, head over to CSS1K.
Connect with ZionCG