Cornerstone Systems is, as of this post, donating LightningDOM to the world — and particularly to the Chain.js and the PURE Unobtrusive Rendering Engine projects, under their respective open-source licenses. LightningDOM was initially created for Project Recon and has been used on some other Cornerstone client intranet projects.
Here is the URL for LightningDOM 1.3: http://www.projectrecon.net/LightningDOM/LightningDOM-1.3.js
Update: LightningDOM 1.3.1 is now available.
Note that LightningDOM currently requires Prototype (1.6.0.2) for classes and inheritance (and maybe some other things), but it should be fairly trivial to make a jQuery version (or lib-independent version) and I hope to do this soon.
Why on earth would you re-implement the DOM?
And in Javascript?? You’ve got to be out of your minds! Well, it turns out there is actually a sane reason: speed. Most web developers know that the DOM is dog slow, particularly in IE-land. There are plans to fix this in IE8, Chrome and possibly Firefox 3.1, but for at least the next few years, we’ll be supporting browsers with a slow DOM.
But is it really too slow? I mean, can’t users wait a couple seconds for the page to load? After all, they’ve been doing it for years! No, they can’t wait. At least, not anymore. Google taught us the competitive advantage of speed and we’ve been trying (though it’s been tough in Web 2.0 land) to not look back.
The first Javascript Ninja I worked with, Ed Ball, taught me to create HTML strings in arrays and then use element.innerHTML = array.join(“”) to pull together the final string and stuff it into the DOM in one fell swoop. This is the most performant way and it’s been a standard client-side developer trick for years. This is usually fine when you’re assembling the HTML from scratch.
But what if you want to write your HTML in HTML, not scattered around your Javascript, so you can use some kind of templating or data-mapping to “inject” data into the HTML? Well, then you find yourself in need of parsing the HTML and mucking about with it. This is tedious, error-prone and regexes are notoriously unreadable (unless you’ve ported Pyparsing to JS). Enter the LightningDOM, a layer between the raw HTML and your templating/data-mapping code, without the slowness of the DOM.
What’s the API? How do I use it?
Unfortunately, the DOM’s API is all properties (.nextSibling, .firstChild), not methods (.nextSibling(), .firstChild()) — and the current version of Javascript doesn’t support getters and setters. So I had a choice. I could either pre-set all the properties of all the objects and updating all the affected properties with every change, or I could slightly break from the DOM API and “functionalize” all the properties.
I chose the latter because I didn’t want to create this huge object graph. Instead, I wanted to use a flyweight pattern where any necessary objects are created on-the-fly and their “functionalized” properties are figured out when requested. You can read more on the architecture in the How It Works section below.
Suffice it to say, the LightningDOM is not immediately interchangeable with the real DOM. You’ll need to change your client code from calling “.attributes” to instead call “.attributes()”, and “.innerText” needs to be changed to “.innerText()”. One option is to write a wrapper around the real DOM to “functionalize” the properties — this way you could switch back and forth between the LightningDOM and the real one. As the real DOM becomes faster in future browsers, this will become more of an issue, however the faster browsers will probably also support getters and setters in JavaScript, thus allowing us to “have our cake and eat it too”.
How do I get started? The first thing to do is construct a new LightningDOM with a string full of markup, like so: var dom = new LightningDOM(strMarkup). Then you can modify your DOM using the standard API, but “functionalized” (adding parentheses “()”) and then dump the result back into the real dom, like this:
<div id='destination'>
<div id='quotes'><span id='favoriteMotto'></span></div>
</div>
var strMarkup = document.getElementById('destination').innerHTML;
var dom = new LightningDOM(strMarkup);
dom.firstChild().innerText("I Love speed!");
document.getElementById('destination').innerHTML = dom.outerHTML();
Update: This example requires LightningDOM 1.3.1
An important note: You don’t have to regenerate all the markup. It is performant to grab just a slice of the markup by doing something like $(‘destination’).innerHTML = dom.firstChild.outerHTML(). I do this in my data-mapping system, which keeps track of the pairing between LightningDOMNodes and JS data objects. When an object has changed. I call dataMapper.update(obj) and it just regenerates the markup for that changed element, thus being faster and preserving any real-DOM manipulations in any surrounding elements.
Note: as of right now, you have to feed it decent XHTML. In non-IE browsers, it’ll display a “failed to parse” alert-box and in IE it should throw an exception on an eval() statement. Every tag has to be either self-closing (“<br />”) or have a paired end-tag (“<p></p>”). The only exception right now is the “<input>” tag, because IE will automatically remove the ending slash for you if you’re pulling the markup from the DOM (no, I’m not joking). So LightningDOM is smart enough to re-insert the ending slash of an input tag. There are probably other tags that IE does this to (<li>, I think?), but I haven’t used these, so I haven’t taken the time to code for them. For the long-term it would be cool if LightningDOM could just intelligently guess when it’s being fed unpaired tags (like BeautifulSoup), but right now it’s not that smart.
How It Works
The LightningDOM parses the given HTML using Regexes. I have two different parse algorithms (but both using the same regex). IE is much faster at parsing the whole markup by doing a single regex replace to convert the markup into JSON and then to eval the JSON. FF (and I’m guessing Chrome & Opera) is much faster at looping through the markup with a global regex.
All the tags are parsed into a two-dimensional array (well, technically an array of arrays), which represents all the information in the tags. Here’s a pretty look at the internal data structure representing a chunk of HMTL (note that we use a simpler object model inspired by ElementTree to representing the inner-text as a “tail” of the previous tag, rather than a nested “Text Node”).
HTML:
<div class="header">
<span id="firstName">Peter</span> <span id="lastName">Rust</span>
Search: <input /> <a href="/logout/">Logout</a></div>
Array of Arrays (_aMarkup):
One of the reasons for this particular architecture is that it can be easily joined into the exact markup with no processing required. This is why we retain whitespace, pointy-brackets, etc — so that .outerHTML() and innerHTML() can be a super-performant join() operation. Note that having a single array might be more performant (you would have to do the tagName extracting, etc on the fly). This is an option I’ve been considering recently.
So… where do the objects come from? The LightningDOMNode objects get constructed on-the-fly, as they’re requested. If you ask for dom.firstChild(), you’ll get back a newly-constructed object with no data — it just has an index pointing to the row in the array-of-arrays that contains its start-tag. The index pointing to the row that contains its end-tag will be calculated on the fly (and cached) if you grab the .nextSibling()
Figuring out the end-tag indexes during the initial parse and storing them in a permanent place (instead of in the throw-away flyweight objects) would probably result in a significant speed increase in many uses, so I’d like to implement this when I get a chance.
Bugs, Warnings & Future Directions
This is beta-quality software. Don’t use it in your company’s 747 auto-pilot program or life-support system. If somebody dies, you can’t sue me. I warned you.
I’ve been using the LightningDOM in production code, so I know it’s stable and predictable with the HTML templates I’m using and the manipulations I’m doing. But I highly doubt it’s bulletproof to the whole range of HTML you could throw at it or the whole range of DOM manipulations you could try. Particularly, I think there may be holes in the end-index cache validation system. If you inject a few elements dynamically, or remove some, it may be possible for a parent element’s cached end-index to not be updated or invalidated properly. Also (as mentioned above), there are particular issues with feeding it HTML from the IE DOM, which strips ending-slashes from certain tags that it may not be able to stomach.
I plan on continuing to use LightningDOM and I would like to continue maintaining it. I hope to do so in the context of Chain.js or PURE or both, but it all depends on whether the project leaders see value in integrating and building on the LightingDOM (or some future version or derivative of LightningDOM).
I am wide open to suggestions for changes from either of these communities, but I will not officially maintain or support the LightningDOM for direct public consumption on this blog. If I do maintain or support it, it will be under one (or both) of the projects listed above and the project leaders themselves will have final say in what changes will be made.
