Breakthroughs In HTML Audio Via Manipulation With JavaScript 141
jamienk writes "Imagine if you could grab and manipulate audio with JavaScript just like you can images with Canvas. Firefox experimental builds let you do just that: crazy audio visualizations, a graphic equalizer, even text-to-speech, all in JavaScript! Work in progress; you need a special build of Firefox (videos available), being worked on via W3C."
Pac-Man (Score:1)
Pac-Man played sound on the Google homepage, wasn't that done in JavaScript?
Re: (Score:1, Insightful)
No, that was Flash, and it caused lots of ire, particularly in combination with Firefox plugin that loaded the Google homepage in the background. I hope there'll be a sound=off option, because I still remember the 90s and web pages with MIDI sounds.
Re: (Score:2)
Humorously, I actually thought the game was silent when I first viewed it. Once again, ClickToFlash delivers an improved user experience.
Re: (Score:2)
Having worked very closely with the source, including walking through the entire audio subsystem, I can assure you that the audio was always flash, and the rest of the game was always JS and Canvas, at least for the first day or so. They may have changed things late in the game.
Re: (Score:1)
It was done via Flash.
Re: (Score:2, Informative)
Re: (Score:1)
Re: (Score:2)
http://www.google.com/pacman/ [google.com]
Flash Helper; State of JS Audio (Score:3, Interesting)
From what I've read, there was an off-screen flash helper playing the audio, and I'm not surprised. It's nigh impossible to get reliable audio with JavaScript only. People have been doing some experiments with stuffing data urls into , , and tags for a while now (heck, I've been doing it since last fall) in order to programmatically generate audio, but it generally suffers from performance issues and various glitches for a while now. You can mostly pull it off only if you stick with MIDI or you use Flash
Should've Previewed (Score:2)
This:
was meant to be
Re: (Score:2)
Have you published your experiments anywhere? I have done some experiments with the audio tag and have been disappointed with the way it handles short programatically triggered sounds. It works (baring bug, like in Chrome) but I would love to see a better way.
js-audio Google Group (Score:2)
Have you published your experiments anywhere?
I talked about them a little bit at the December BayJax Meetup, and I've been meaning to turn that presentation into a web page since then, but I haven't.
However... I've also been meaning to create an audio-focused JavaScript mailing list so that when I do put it up, I'd have somewhere to announce it. And thanks to your comment, I just got off my butt and created the js-audio Google Group. :)
http://groups.google.com/group/js-audio [google.com]
I'd love to have you or anyone el
Re: (Score:3, Informative)
There is no Flash involved here, you read wrong. I wonder why you got modded up.
Re: (Score:2)
There is no Flash involved here, you read wrong.
In the topic of the linked article, or in Google's Pac-man?
I wonder why you got modded up.
Possibly because even if it turns out what I've read about GPac is incorrect, most of what I've written besides is correct.
Dear Internet: Sorry. (Score:4, Funny)
Of course this has good uses (blind users with scripts available), but I can see how this will end badly for the rest of us.
Re:Dear Internet: Sorry. (Score:5, Funny)
You mean annoying like people who put their comments between <code> tags?
Re:Dear Internet: Sorry. (Score:5, Funny)
Re: (Score:1, Offtopic)
Goddamn it, why don't I ever have mod points when I need them?
Re:Dear Internet: Sorry. (Score:5, Insightful)
IE doesn't even have support for canvas, Firefox has had it from 3.0 at least, and I think even 2.0 had some support.
If IE still has more than 30% worldwide marketshare, and doesn't have basic requirements for this, its not going to be used. Period.
Canvas has been around for ages and is there even a single practical example on a site people use daily? Yeah, there are about a million tech demos but very little actual use because IE doesn't support it.
Re: (Score:2)
Re: (Score:2)
IE9 will have Canvas
Re: (Score:2)
Re: (Score:2, Funny)
You know, there are already PLENTY of applications out there that blind users can use to turn text on screen to speech? And that a lot of people prefer to disable javascript from running?
Last thing we need is an "on Hover" Javascript ad, you mouse over, and audio gets maximized "FIND YOUR MATCH @ ADULTFINDER.COM!"
Re: (Score:2)
Will it be used? (Score:2)
Until IE implements even half these features, we will be stuck with "Quick"Time and Flash for quite some time...
Re: (Score:2)
> Until IE implements even half these features...
Or until IE is dead...
Re: (Score:2)
IE is already dead as far as developers should be concerned. Pages will work well enough that IE users can still limp along, if need be.
That's just (Score:2)
Screen readers... (Score:3, Insightful)
HTML 5 is AMAZING!!! (Score:1, Insightful)
awesome, you could do that in Flash 5 years ago.
Re: (Score:2)
Re: (Score:2)
The difference is that we want flash to die horribly for a number of reasons.
Re: (Score:2)
Re: (Score:2)
With no way to fine-tune who is allowed sound and who isn't.
With a browser implementation of an open standard, it would be easy to add features such as muting a page's audio when (*gasp*) it's not in the foreground tab. Besides, there will no doubt be plenty of ways to turn off audio, partially or completely. And plug-ins like greasemonkey work great to hack around website JS features, while you can't do anything with Flash SWF blobs other than turn them on and off.
Flash is AMAZING!!! (Score:2)
awesome, you could do that in Flash 5 years ago.
awesome, you could do that in a Win32 app 15 years ago.
Administrators group (Score:2)
awesome, you could do that in a Win32 app 15 years ago.
You have to be a member of the Administrators group to install a native app. You do not have to be a member of the Administrators group to run a Flash object, a JavaScript program, or a Java applet, because the web developer can safely assume that every PC already has players for one or more of these.
Re: (Score:2)
A music visualizer needs to be installed? When did Windows prevent normal users from just running executables?
Windows can also mount /home noexec (Score:2)
When did Windows prevent normal users from just running executables?
Software Restriction Policies [microsoft.com] started in Windows XP. This is similar to the UNIX feature of mounting /home noexec.
Re: (Score:2)
html 5 is screwed from the start because it isn't xhtml2.
If I was to design the future of the web, it would be xhtml2 with css3. Plus simple site based local storage (cookies that don't always get transmitted to the server would do if the storage size policy was a little less strict), the ability to transfer part of a model to the next page on submit without making a round trip to the server, and throw in some canvas/audio/video tags for everyone else enjoy.
Re: (Score:2)
If I was to design the future of the web, HTML interpretation would be just one of the many applications the web browser would be able to process. The browser would provide a virtual machine environment, allowing applications to be executed (not unlike Javascript is today, but a richer environment). If you want to display HTML, you can write/link to an HTML rendering engine able to run within the environment.
HTML has its place, but it is seems incredibly silly to write applications using HTML and CSS.
Re: (Score:2)
html 5 is screwed from the start because it isn't xhtml2.
If I was to design the future of the web, it would be xhtml2 with css3.
Thing is, people tried that, and it failed horribly. HTML5 is actually working, because it emphasizes compatibility. If you want to live in your utopian pipe dream and throw out everything to start from scratch instead of hacking on top of the mess we're stuck with, have fun, but the proof is in the pudding. XHTML2 died, HTML5 has won.
Maybe it's because browser implementers are evil and not forward-thinking enough, but guess what? Step one to a successful standard is get implemented. No browser imple
I Can't Wait! (Score:2)
even text-to-speech, all in JavaScript!
WooHoo! I'm gonna have hidden fields in my web pages so this "text to speech" will say one thing while the text on the screen will say another!
don't say you weren't warned: (Score:2, Interesting)
the future os is the browser
the future os programming language is javascript
enjoy!
Re: (Score:1)
You need to put "enjoy" in quotes.
Re: (Score:2)
sure, but that's because the marketplace for browser-based languages is immature. As it displaces al other forms of programming platforms, Microsoft will bring out javascript.net, mozilla will bring out xpjavascript, and Google will bring out javavik.
See, the future will turn out to be just as good as today.
Re: (Score:2)
The future sucks badly.
Hopefully, the world will end in 2012.
Re: (Score:2)
I don't buy it. It's too inherently limited to replace any large class of regular applications.
I fully expect the web-app trend will be just that... every bit as short-lived as every other stupid trend. This time it's Flash and Javascript. Last time it was Java Applets. Next time... who knows?
Re: (Score:2)
Python already has a DOM manipulation library (in fact, I think it has 2 or 3 which could be used in the normal distribution), the problem is browser support. If browsers don't use it, then web devs can't use it.
irrelevant (Score:1)
LoB
wisdom (Score:2, Insightful)
Re: (Score:2)
and why in this case is it "not to be done"?
Re: (Score:2)
Off-Topic: Fix the Blog Search Form (Score:2)
The last missing piece... (Score:2)
...that was needed to be able to actually make a decent 3D game in JavaScript. Sure, I'd prefer OpenAL bindings, but this'll work.
Re: (Score:2)
Re: (Score:2)
WebGL is hardware-accelerated video. Audio benefits from hardware acceleration, but doesn't need it. It does, however, need some amount of software control, something more than "Play this file at roughly this time."
Re: (Score:3, Insightful)
FYI, Quicktime is not awful at all on Mac OS X, nor is Safari and iTunes. And almost everything from Microsoft and Adobe sucks on Mac OS X.
I don't know why you associate Quicktime with online audio, so what you probably meant to say was "The quicker we can get away from our reliance on that god awful Flash, the better."
Be prepared for another "Firefox vs the World" with this, however: Vorbis vs MP3/AAC.
Re: (Score:1)
We beat Microsoft and Apple once before. Open Source can do it again.
Re: (Score:1, Troll)
What do you mean by "We beat Microsoft and Apple once before"? The market share for both Firefox and Opera is stable but the marketshare for Safari and Chrome is going up.
Firefox is bloated, I for one don't want to use it. It's either Safari, Chrome or Opera for me.
Re: (Score:2)
The marketshare for Safari is only going when new iPhones or iPads come out. If you look at the w3schools.com listings, you'll see that when new Apple products aren't coming out, the marketshare for Safari actually went down from 3.8% to 3.7%, whereas the number a year ago was a little over 3%. Considering the number of Apple handhelds entering the market, you'd think the number would be going up faster than that.
In the same period that Safari went fro
Re: (Score:2, Informative)
For fuck's sake, when will people understand that the listings on w3schools.com ARE ONLY FOR THAT SITE AND NOT REPRESENTATIVE OF THE GLOBAL MARKET SHARE?!
Re: (Score:2)
The market share for Firefox is far from stable.
Here [mozillazine.org]. If you dig a little on that blog you'll find the actual sources, which are a mix of reliable web statistics providers.
Re: (Score:3, Informative)
Not really. Vorbis has about the same quality per bit as AAC (unlike theora vs h264), and it's established long enough to not have patent issues. There's no reason not to implement support for Vorbis, and it's plenty good enough to be the default codec. What's more, Youtube's behind Vorbis (it's part of the WebM spec), and since Flash has pledged support for WebM, they'll have it too.
Re: (Score:2)
And from what I've read, there is no-floating-point software decoders that don't require much CPU power either, so software-only solutions won't drain the battery significantly. I don't doubt for a second that Opera would also support Ogg Vorbis. The problem would be to get Microsoft and Apple to add support to their products.
However, if we look at the past:
- Microsoft: BMP screen captures (Microsoft format)
- Apple: PNG screen captures (Open format)
- Microsoft: Windows Media Audio/Video (two Microsoft forma
Re: (Score:2)
Re: (Score:2)
Do you have reason to believe that Apple will include Vorbis support for Safari but MS will not support it for IE12?
As a content creator, I really like Ogg Vorbis. The limited number of devices and apps that support it is a drag. I hope it becomes more widely accepted. I
Re: (Score:2)
My reason to believe this is the two lines you didn't quote from my post:
- Microsoft: BMP screen captures (Microsoft format)
- Apple: PNG screen captures (Open format)
If Apple was so anti-open-formats, they would never have picked PNG as the default format for screenshots. Microsoft has a history of always creating their own formats even when a multitude of alternatives already exist (kind of like Sony with their half a dozen Memory Stick types).
But I guess Apple could have used FLAC instead of creating Appl
Re: (Score:3, Interesting)
Apple picked PNG because Microsoft didn't. And screencap image formats were hardly going to be an industry deal breaker. BMP files are so obvious as to really not give any leverage to Microsoft. Any patent on the silly thing would be overturned("Method for displaying a picture by lining up the colored pixels in order?").
Apple is pro-open-formats only if there's no way to get a lock-in. Apple is a patent holder with MPEG-LA, hence their preference for H.264.
Remember that there's nothing preventing you fr
As bad as patent law is now (Score:2)
Android Pod Touch (Score:2)
Now if Sansa would only come out with a large storage device that has a screen at least as big as a Zune (and hopefully as big as an iPod Touch).
If you want an "Android Pod Touch", get an Archos tablet.
Re: (Score:2)
Microsoft: Windows Media Audio/Video (two Microsoft formats)
Microsoft has already pledged H.264 support (and AAC for audio) in IE9 out of the box, and WebM (VP8+Vorbis) if codecs are installed.
Re: (Score:2)
I brought it up to show that when Apple needed a file format for the screenshots of Mac OS X, they went with PNG (which is open like Vorbis) while Microsoft created BMP.
Re: (Score:2)
Re: (Score:2)
PICT [wikipedia.org]
Re: (Score:2)
Why didn't Microsoft simply use TIFF or TGA for the graphic format? Why invent BMP at all?
Re: (Score:2)
Vorbis has about the same quality per bit as AAC (unlike theora vs h264), and it's established long enough to not have patent issues.
Age is not an issue, MPEG-2 still has patent issues! The issue is that Vorbis has not been adopted by enough deep pockets for the patent trolls to come out yet. Already they are licking their lips over VP8.
Re: (Score:3, Informative)
Re: (Score:2)
Indeed, Activision Blizzard, with revenues of $4 billion last year, is big enough for patent trolls....if they actually use Vorbis.
Re: (Score:2)
The current generation of AAC (HE-AAC/aacPlus/etc) is undeniably vastly better than Vorbis. In fact, the parallels with H.264 are stunning. HE-AAC is part of the MPEG-4 standard, it's not particularly impressive at high bitrates, but vastly superior at low bitrates, etc.
The only difference is that people have been rushing to put H.264 everywhere, while they've been reluctant
Re: (Score:2)
Also worth pointing out that the majority of HTML5 browsers already supports both Vorbis and Theora. I think Safari is the ONLY HTML5-using browser that doesn't have support for Vorbis out of the box. I believe Chrome ALSO supports vorbis and theora (not just aac/h.264), and between Firefox and Chrome that's got to be at least, what, roughly 60-80% of HTML5-supporting browser traffic?
Since Apple® Quicktime® can actually be taught to understand Vorbis, Theora, and all of the other Xiph codecs syst
Re: (Score:1)
Mod this up. I can't remember the last time I used quicktime since flash came along.
Re: (Score:2, Informative)
Further more, Quicktime's browser plugin assigns itself as the default player for many types of media - including mp3 files. So every time I click on
Re: (Score:2)
Aren't trailers from that website encoded in H.264 video, AAC audio in a standard MP4 container? Shouldn't that play on anything from Windows Media Player to VLC?
Re: (Score:2)
Re: (Score:1)
Re: (Score:1)
I thought at first that themp3 standard [digitalpreservation.gov] was already "free":
Open standard. Developed by the Motion Pictures Expert Group (MPEG), Coding of audio, picture, multimedia and hypermedia information.
However, mp3 is not free [mp3licensing.com]...yet. Some of these patents are set to expire on their 20 year time frame in a couple of years it would seem.
Re: (Score:3, Informative)
> However, mp3 is not free...yet. Some of these patents are set to expire on their 20 year time frame in a couple of years it would seem.
Yes, the next MP3 patent expires this Sunday. The longest patent seems to expire in 2018 but that appears to be MPEG-2 LSF and only required for low sample rate MP3s. So the next furthest date looks like April 2017 but it may be worth double checking the dates on those around 2014/2015.
http://wiki.multimedia.cx/index.php?title=MP3_Patents [multimedia.cx]
Re: (Score:2)
Yes it would seem still no native mp3 support.
The real fundamental question is, however, do we really want to use java for audio?
mp3 is a patent-encumbered standard. We don't want any of that in your open standards, now do we? Also, javascript is not java; it really has nothing to do with java, except the ill-chosen name.
Re: (Score:1)
Also, javascript is not java; it really has nothing to do with java, except the ill-chosen name.
Valid point...I see where the 'Offtopic' tag came from now, thanks :)
Re: (Score:2)
Re:Firefox, eh? (Score:5, Informative)
Third, unlike Theora and H.264, Vorbis is actually much better than MP3.
Vorbis vs MP3 (Score:2)
In all honest ignorance and curiosity, how so, aside from the patent legal issues? Are the file sizes smaller? Is playback processor load less? Is playback quality at a set sampling rate audibly better? Etc., etc.
Cheers,
Re:Vorbis vs MP3 (Score:5, Informative)
Vorbis has smaller file sizes for the same quality or better quality for the same file size (q1 (~80kbps) is easily comparable to 128kbps MP3).
Vorbis also supports up to 255 channels with official channel ordering specs up to 8 channels (7.1), while MP3 only supports mono and stereo.
Worth it to transcode library of MP3 files? (Score:2)
Good to know, thank you for the reply.
I have a sizable existing library of MP3 files. Would it be worth the trouble to transcode these into Ogg? Would this process be problematically lossy? And will iPods will play Ogg files? (This last is curiosity; I don't use iTunes, so it's not too much trouble to wipe my iPod and put something in it that plays Oggs.)
Cheers,
Re: (Score:2)
IMHO, it is never worth converting audio files from one lossy format to another. Suppose you do that, and your files are now half the size but 1% noisier. But by the time you have re-encoded your 5 bzillion gigs of data, storage prices have dropped in half. So you could have had better results by sitting around doing nothing.
Re: (Score:2)
Re: (Score:2)
Actually, wtfjs.com is all about syntax quirks and nothing about the current APIs being an "amalgamation of junk" - and some of them are just "proof of concepts", not real problems, like using a reserved keyword as a variable name.
Besides, who is doing videoElement.play() or videoElement.pause() bad? It's a very clean API, imho.
Re: (Score:2)
And I'm guessing that there is absolutely no point to this audio manipulation via "JavaScript" when it only works with one specific build of one specific browser's particular JavaScript implementation.
It's called a proof of concept.
Re: (Score:1)
Re: (Score:1)
Sounds like the lyrics for a future Weird Al Yankovic song.