HTML5 Audio Karaoke – a JavaScript audio text aligner

June 1, 2012 | Bible Tools, HTML/CSS, JavaScript | 23 Comments

What it Does

Based on some amazing work by my friend Weston Ruter, I’ve put together a little library that mashes together

  1. some text (usually some HTML)
  2. an audio source reading that text (usually an mp3)
  3. a timing file (in this case, generated by CMU Sphinx)

The result is that when you press “play” the words are highlighted as they are read, and you can click on words to navigate through the audio. The magic comes from data produced by the CMU Sphinx library (based on Weston’s work) which creates the word timing information.

I put together two demo versions, one of Martin Luther King, Jr.’s I Have A Dream speech and another one of the English Bible using the English Standard Version which has as great API. Unfortunately, the MLK speech didn’t align very well so the demo isn’t very good other than as an example of how dependent the process is on a good alignment.

(note: right now it’s Chrome/Safari/IE9 only since it requires MP3 playback)

How it Works

Although I wanted to use a “standard” format like WebVTT, I also wanted the filesize to be compact since my intended project involved large datasets of 48 hours or more of audio (i.e. the Bible). So here’s the basic JSON format:


Basically, it’s just an array of words with a start and end time. The array of arrays format is quite a bit smaller than using JSON and doesn’t require any processing like WebVTT (although that might change later). It would take quite a bit of time to produce something like this by hand, but Weston used the CMU Sphinx library to generate this data, and it’s probably been about 90% accurate for the entire ESV Bible.

Once all the data is loaded, the AudioAligner class searches through a DOM node for the words in the array, skipping over classes or tags you define, and then links those words to the audio player.


Again, the demo I put together utilizes the API provided by the creators of the English Standard Version (ESV) of the Bible. The API allows developers to request the text and the MP3 and then this is mashed up with the timing files generated with SMU Sphinx.

HTML5 Karoke Demo

If anyone’s interested in the library, please let me know in the comments and I’ll post it to Github.

Cross-Browser Native Get/Set Properties in JavaScript

May 2, 2012 | JavaScript, Video | 10 Comments

JavaScript Doesn’t Support Properties

When I first created MediaElement.js a few years ago, I wanted to make a JavaScript object that fully mimicked the HTML5 Media API, but under the hood it might have a true <video> tag or a Flash object doing the rendering.

The problem I quickly found was that the <video> tag has several properties like .src and .volume that can’t be replicated since JavaScript doesn’t have true get/set capability that works in all browsers (by “all browsers” I mean “IE6 and up”). So I had to create methods like .setSrc() and .setVolume() (or use jQuery’s .volume() syntax) to make it work correctly across browsers, but that resulted in a API that didn’t match the HTML5 spec.

Ah, But JavaScript Does Support Properties

A long time ago, in Firefox’s distant past there was a proprietary way to create properties, using the __defineGetter__ and __defineSetter__methods. Here’s what it looks like:

var myObject = {};
myObject.seconds = 22;
myObject.__defineGetter__('milliseconds', function() { return this.seconds / 1000;} );
myObject.__defineSetter__('milliseconds', function(value) { this.seconds = value * 1000;});

myObject.milliseconds = 1750;
console.log(myObject.seconds); // outputs 1.75;

It’s great because it allows you to easily define properties that can do additional calculation work (e.g., from milliseconds to seconds), but it doesn’t work in all other browsers. The method that is more universally available is Object.defineProperty which offers a slightly cleaner syntax:

var myObject = {};
myObject.seconds = 22;

Object.defineProperty(obj, 'milliseconds', {
     get: function() { return this.seconds / 1000;},
     set: function(value) { this.seconds = value * 1000;}

myObject.milliseconds = 1750;
console.log(myObject.seconds); // outputs 1.75;

Looks super useful right? So why doesn’t anyone use it?

The problem is that IE6 and IE7 don’t support it, and while IE8 does support the Object.defineProperty method, it sadly only works on DOM objects that are attached to the tree.

So, if you really, really needed an object that supported properties, you can create a DOM object, attach it to the document, and then use Object.defineProperty to give it some properties. That’s a lot of trouble just to support IE8, and you still don’t get IE6 or IE7 support, so I don’t know of any libraries that currently use the approach.

Hacking IE6 and IE7

I recently came across a technique that I haven’t seen demonstrated before from Jonathan Neal using the onpropertychanged event. IE will fire this event when any property, native or added by a developer, is changed. However, like IE8 this technique only works on a real DOM object, not a plain old {}.

Based on Jonathan’s work, here is a function that will add a property and work in IE6 and up.

// Super amazing, cross browser property function, based on
function addProperty(obj, name, onGet, onSet) {

	// wrapper functions
		oldValue = obj[name],
		getFn = function () {
			return onGet.apply(obj, [oldValue]);
		setFn = function (newValue) {
			return oldValue = onSet.apply(obj, [newValue]);

	// Modern browsers, IE9+, and IE8 (must be a DOM object),
	if (Object.defineProperty) {

		Object.defineProperty(obj, name, {
			get: getFn,
			set: setFn

	// Older Mozilla
	} else if (obj.__defineGetter__) {

		obj.__defineGetter__(name, getFn);
		obj.__defineSetter__(name, setFn);

	// IE6-7
	// must be a real DOM object (to have attachEvent) and must be attached to document (for onpropertychange to fire)
	} else {

		var onPropertyChange = function (e) {

			if (event.propertyName == name) {
				// temporarily remove the event so it doesn't fire again and create a loop
				obj.detachEvent("onpropertychange", onPropertyChange);

				// get the changed value, run it through the set function
				var newValue = setFn(obj[name]);

				// restore the get function
				obj[name] = getFn;
				obj[name].toString = getFn;

				// restore the event
				obj.attachEvent("onpropertychange", onPropertyChange);

		obj[name] = getFn;
		obj[name].toString = getFn;

		obj.attachEvent("onpropertychange", onPropertyChange);


// must be a DOM object (even if it's not a real tag) attached to document
var myObject = document.createElement('fake');

// create property
myObject.firstName = 'John';
myObject.lastName = 'Dyer';
addProperty(myObject, 'fullname',
	function() {
		return this.firstName + ' ' + this.lastName;
	function(value) {
		var parts = value.split(' ');
		this.firstName = parts[0];
		this.lastName = (parts.length > 1) ? parts[1] : '';

console.log(myObject.fullname); // returns 'John Dyer'

Pretty aweseome.

Real World Application

Mozilla’s April Dev Derby was on using the <audio> tag, so I thought it’d be a fun chance to try this out and make a native looking HTML5 API that could wrap things like a Flash Ogg player or the amazing JsMad library which can play MP3s using pure JavaScript (great for Firefox which can’t play MP3s natively).

The result is a fun little audio library called Shimichanga that has functional properties like .src. What’s interesting about the library is that when you set the .src property, the set method loads an appropriate renderer (HTML5, Flash, or JavaScript) depending on the browser’s capability and then sends the value to renderer to use, all seemless to the developer and end-user. Note: Mozilla didn’t allow the flash shim part, so to see a full working demo go to:

Hopefully, I can back port this into MediaElement.js and make the entire library more dynamic and usable. For now, go check out my hideously styled demo on Mozilla’s site (Shimichanga) and give me some likes!

Dev Derby

HTML5 Video Wrapper for YouTube and Vimeo API – MediaElement.js

November 28, 2011 | JavaScript, Video | 51 Comments

YouTube and Vimeo APIs

YouTube and Vimeo have nice APIs to allow JavaScript developers to control the playback of embedded content (oh, and YouTube has a new design). They’ve also updated the APIs to use the newer <iframe> embed style instead requiring <object><embed> flash tags. But as powerful as those APIs are, they are not consistent with each other and neither one conforms to HTML5 <video> properties or events so you can’t leverage your existing skills or code.

MediaElement.js HTML5 Wrapper

To make the YouTube and Vimeo APIs easier to use, I wrapped the MediaElement.js shim structure around their proprietary APIs to make them feel like HTML5. So for a YouTube video, you can use this <video> markup

<video id="youtube1" width="640" height="360">
	<source src="" type="video/youtube" >

Then to build a full-fledged player that is CSS skinnable on top of YouTube Chromeless , you can simply call the MediaElementPlayer like this:

jQuery(document).ready(function($) {


Or you can skip MediaElementPlayer’s controls and build your own player using just the MediaElement wrapper which does not require jQuery (MediaElement is the wrapper which shims HTML5 support into old browsers, MediaElementPlayer is the full-fledged jQuery-based control bar built on top of MediaElement).

new MediaElement('youtube#', {
	success: function(media, domNode) {
		// add HTML5 events to the YouTube API media object
		media.addEventListener('play', function() {
			console.log('yeah, it is playing!');
		}, false);

		media.addEventListener('timeupdate', function() {
// access HTML5-like properties
			console.log('current: ' + media.currentTime);
		}, false);

		// add click events to control player
		myMuteButton.addEventListener('click', function() {
			// HTML5 has a "muted" setter, but we have to use a function here
		}, false);

Once the success event fires, the media object is a JavaScript wrapper that mimics the HTML5 Media API, but under the hood is really a wrapper around YouTube’s API. Nice right?


There are a few things you should look out for if you want to try it:

  • This code is now in the official 2.4.0 release of MediaElement.js, but it should be considered experimental. I’d like to pull it out of the core and make it a plugin, but this will require some re-architecting.
  • The Vimeo API wrapper is not finished yet. It will simply display the default Vimeo controls. (Also, unlike YouTube’s chromeless option, only Vimeo Pro users can totally remove controls)
  • Some browsers (Chrome, IE) don’t allow HTML to be placed over an <iframe> with Flash inside which makes MediaElementPlayer’s controls not work, so I’m using the pure Flash version of YouTube for desktop browsers
  • Fullscreen: JavaScript can’t initiate Flash’s true fullscreen, and Firefox still has terrible Flash support (if you try to adjust the size of a Flash movie’s surrounding DOM nodes, Firefox reloads it!)

Demo Download

Pinbox.js – Photos Then and Now (jQuery plugin)

November 2, 2011 | JavaScript | 6 Comments

With the help of a colleague, I’m gathering images of an area of Dallas, TX just east of downtown around the campus of Dallas Theological Seminary where I work. I wanted to make the photos interactive like the animated GIF you see here.

Live Demo Download Code

Code Explanation

Continue Reading…

Native Fullscreen JavaScript API (plus jQuery plugin)

October 20, 2011 | JavaScript | 91 Comments

HTML5 <video> is great, but when it was first released, one of the big complaints was that it couldn’t do true FullScreen like Flash. Thankfully, this is changing and native FullScreen support is coming to most browsers in the next few months (no word from the Internet Explorer team Update on IE below #5))

The API is still heavily in flux especially since the W3C joined in this week. I spent some time working through the differences to implement FullScreen in MediaElement.js HTML5 video player, and it’s working great in Safari 5.1+, Chrome Canary Chrome 15+, or Firefox Nightly (go to about:config and set full-screen-api.enabled= true) and scheduled for Firefox 10. Below I’m going to try to explain how things evolved, where we are today, and then some code that you can play with.

Simple Demo Video Demo Continue Reading…

Hi, I'm John Dyer. In my day job, I build websites and create online seminary software for a seminary in Dallas. I also like to release open source tools including a pretty popular HTML5 video player and build tools that help people find best bible commentaries and do bible study. And just for fun, I also wrote a book on the theology of technology and media.

Fork me on GitHub

Social Widgets powered by