a web-based karaoke prototype as technology feedback loop

Dan Mosedale

 

Thanks to several excellent contributors and up-and-coming web technologies (most critically Rainbow and popcorn.js), it’s now possible to record yourself singing along to a fine accordion rendition of “Amazing Grace” by Jono Xia, a feat that would have once required Flash.

Here’s a screencast of my friend Courtney Hatt joining in on vocals:

She recorded the vocals directly in the browser using a webcam exposed to the web-page using Rainbow.

Even better, install Rainbow 0.4 (currently available on Mac and Windows) and sing along at theband.mozillalabs.com. If you run into any issues, see the README on GitHub for more details. Mac users should keep in mind that they’ll need to restart their browser from the command-line as described on the install page for Rainbow to work.  In general, using headphones while you’re recording will result in much better sounding playback.

how The Band came to be, and why

My intent here was to prototype something that would be a compelling example of emerging Mozilla technologies in a way that could help the developers of those technologies by acting as a part of a tight feedback loop (in the spirit of the Firefox JS audio API experiments led by humph). I also wanted a prototype that invited significantly more experimentation around further technologies and use cases.

the first cut

The idea came after seeing Rainbow‘s ability to record audio & video, and popcorn.js subtitling. I got a page up and limping, which got positive and excited reactions from a few people that I showed it to. So I then iterated on it to the point of making it more real and easier to use, though it’s still very much a prototype (gotta love that inline CSS and lack of tests!).

One important aspect of karaoke as a choice for this sort of demo is that it was clearly simple enough to be stood up in a basic way fairly quickly, but it would feel compelling to users, even at the beginning stages.

demo as test bed & feedback loop

The first version of The Band has already yielded dividends in terms of helping new technology evolve and mature. This particular use case bumped up against the edge of the recording APIs in ways that evolved them to be more flexible, and Anant’s asyncifying of those APIs (available now as part of Rainbow 0.4) was key. He also did a great job improving the user-experience of asking for permission access the camera and microphone. Implementing The Band also surfaced a few minor bugs in popcorn.js (whose team was also very responsive in helping figure out what to do about them).

Folks who have one of Mozilla’s Open Web Apps prototypes installed will notice that The Band can install itself as a web app, and code has already been written that allows snapshots from the video to be sent to a photo-sharing service using the still-under-development web-app services API (though that’s not yet exposed).

further experiments

As I’ve been working on and talking about this demo to folks, it’s quickly become clear to me that there are lot of useful and interesting possibilities, both to make The Band more fun and compelling to users, and to help evolve various Mozilla technologies. A few of the possibilities that have come to mind so far include:

  • use Rainbow to capture video of the combined recording and publish it to video sharing services, using open web app or other sharing APIs
  • flesh-out interactions between various evolving APIs: audio/video capture (Rainbow), processing (extended audio), and effects (WebGL, processing.js)
  • evolve use-cases, APIs, and code for popcorn.js and Butter themselves
  • provide a testbed for forward-thinking musicians to experiment with various monetization scenarios using web apps

So far, this is purely a prototype.  If and how it moves forward remains to be seen.

feedback

is happily accepted on this labs mailing list thread.

the code

is on github. Fork and enjoy!

credits

Thanks especially to Anant Narayanan for Rainbow and his work on evolving its APIs, to Brian Brennan for his visual style facelift, Atul Varma for pairing on some of the apps stuff, and Jono Xia for his accordion stylings, Courtney Hatt for her vocals, the popcorn.js, Open Web Apps,  jQuery, and Noisy teams.