In June 2013, we published the Extensible Web Manifesto, declaring that the web platform should be built a series of extensible layers. Low-level capabilities, like hardware access, cryptography, or the parser, should be exposed to web developers through JavaScript, and higher-level features, like HTML tags or animations, should be explained in terms of those primitives. This allows JavaScript developers to extend the web platform without rewriting it from scratch in JavaScript, customizing it for the needs of their applications and creating a virtuous cycle wherein web developers prototype higher-level features that implementers can later pull in.
In this talk, I want to explore the concrete meaning of the extensible web. What initiatives already under way can we see through this lens? What ideas are still gestating, and need your input as a web developer? How will this new philosophy end up impacting you? I want to show you the future of the web platform, where developers like you are involved from the beginning in designing and prototyping APIs, and have enough low-level tools at your fingertips to solve any problem you face. And I want to bring it all back to our favorite language, JavaScript, which is the glue that makes this all work together.
In text format at http://domenic.me/2013/10/07/the-extensible-web/
The web platform has, historically, been a kludge.
It’s grown, organically, into a mess.Our platform has no sense of cohesionMost of its APIs have been poorly designed, by C++ developers, via a binding layer meant originally for CORBA.There have been major gaps in what we can do compared to native appsAnd for those things we can do, we end up doing them by drowning ourselves in custom JavaScript functionality
Generally, new things have been introduced into our web platform via:Months or years of mailing list standardization, writing something in prose and IDL, driven by scenario-solving without much concern for actual usabilityThey expose some fundamental capability in terms of a high-level API or declarative form that burrows down directly to the implementer layer of C++, with limited customizabilityAfter all this time, it eventually ends up in your hands, and you end up telling the standards bodies that it’s a huge mess, that it solves half your problems half of the time.
(Heh. Most successful platform ever!)
We wrapWe prolyfill (explain)We transpileAnd in one case, promises, we even built an interoperable standard from the ground up.We need our platform to be better, and so we make it better ourselves.
We’ve been shouldering all the work until now, writing massive JavaScript libraries or transpilers to reinvent basic functionality.There’s a better way, where we work together toward the future.What these standards bodies have realized is that…
This means two things:Explaining features of the platform that are already there.Wouldn’t it be weird if we had compound words like “scifi,” but didn’t have any word for “science” or “fiction”? If some standards body, perhaps the French Making Up Words Consortium, just handed us the word “sandpaper,” but we had no way in our language to talk about “sand” or “paper” individually? The web is like that today, and we’ll see a few examples.Giving you new low-level features that you can use.If you want to invent the word “scifi,” somebody had better have come up with words for “science” and “fiction”! Similarly, there’s lots of things we just don’t have “words” for on the web, yet. That’s where native apps are hurting us.So with this in mind:
The most fundamental gap in the platform that’s unexplained: how do those darn tag things even work?Somehow, you feed a string containing some angle brackets into the browser, and they get turned into these JS objects with terrific APIs, which we call “the DOM.” How does that even work?Custom tags explains this process, saying that you register a mapping of tag names to element prototypes with the browser, and that’s what the HTML parser is actually doing under the hood. This is great! The democratization of HTML!And better yet, no more crazy widget libraries, with their own semantics. No more jQuery UI with this .option thing, no more Dojo dijits, no more Bootstrap craziness, no more WinJS with their funky winControl property. Just tags, that turn into elements, which behave like normal DOM things: they have properties, getters, setters, methods, you name it!
But what about the existing tags? Half the reason these widget libraries exist is so that you can create your own freakin’ <select> element, because otherwise it’s not styleable or customizable.In general: you know how you have these magic tags, like <select> or <input type="date"> or <details> or <video> even good old <li>, where it looks like there’s some extra “stuff” the browser’s doing, but you have no idea how?Yeah, that’s the shadow DOM.But what’s great, is that once we actually have a basis for these hidden parts of the DOM, in reality instead of in C++ magic land, you can actually start hooking into them instead of rebuilding an entire element just to customize its behavior and styling.“I'm working on transitioning the <select> element from a custom renderer to shadow dom.”
This is a good example of both adding new capabilities and explaining old ones. Because you can do fundamentally new things with web audio, like positional audio or audio synthesis or so many other cool things.But remember the <audio> tag, from way back in 2009? Well, it’s kind of the quintessential “here’s some C++ magic thrown over the wall to you guys.” From an extensible web perspective, the <audio> tag should be explained in terms of web audio!
But speaking of fundamentally new capabilities, you can find examples of such things popping up all over the platform.
ES6:Extensibility of built-insProxiesES7Object.observeWeak references---So now we have a capable platform, with all these new low-level capabilities, and with our existing capabilities actually composed out of nice low-level primitives. This means we can stop rebuilding the entire platform just to customize one piece.
The second half of this extensible web philosophy is that we need to tighten the feedback loop between developers and standards bodies.Because think about it: you have all these great things that you’ve built out of our low-level tools, but now you’re downloading megabytes of JavaScript or transpiling your code just to get the base of your platform. This is why almost every web page uses jQuery: because the platform itself hasn’t stepped up to the plate and incorporated jQuery’s innovations back in.In short, we need to incorporate this kind of slang you’ve invented back into our shared language.
Adopted by jQuery, Dojo, Angular, Ember, WinJS, YUI, … it’s time to put these in the platform.
FileReader, XMLHttpRequest, getUserMedia, postMessage, object URLs, MediaStreams… just like with promises, we’re missing a piece of slang here.Node.js has led the way, but there’s still some lessons learned with their implementation that could make our life easier.In the end, we want to be able to take various sources (URLs, camera data, already-downloaded binary payloads like videos etc.) and pipe them into various sinks (<img>, <video>, and <audio> tags; other <iframe>s; custom consumption code).
The basic act of doing an HTTP request has so much complexity: cross-domain protection, redirect following, deserialization from bytes, cookie jars, caches… We want to provide the basic building block, and the ability to layer each of these features on top of it.
Active investigation going on into how to expose compression primitives to the web
And what else? What do we need? What is preventing you from building webapps of your dreams? You tell us!
The best thing you can do to get involved in these things is to prolyfill.There’s only so much standardization bandwidth to go around, so if you can create a de-facto standard like jQuery, or an open specification like Promises/A+, the world is waiting.If you want to figure out what a zlib API for the browser should look like, the best thing you can do is:Learn what the constraints and use cases are (and not just your use cases, but everyone’s!)Design an API and library to prolyfill this gapEvangelize its use among developers, so that everyone recognizes it as the clear solution that all developers just want browsers to ship.
The W3C Technical Architecture Group had four seats go up for reelection recently, and four “reformers” were elected at once: Yehuda Katz, Alex Russell, Marcos Caceres, and Anne van Kesteren. The extensible web philosophy underlies their governance, as the ultimate technical body which provides guidance and approval for all W3C specs. We’ve already seen fruit here with their reviews of the web audio spec, among others, where they have helped these specs build a solid grounding in JavaScript fundamentals and generally be less magic and more JavaScript. All their work is being done on GitHub, as are more and more specs. This is happening.
More generally, if you want to be involved in helping the web succeed by guiding us toward better standards, then let’s talk. It’s an area I’ve been diving into over the last half-year, stemming from my Promises/A+ work but expanding into many other things. Finding the right approach and content is delicate, as these people are used to noobs coming out of the woodwork to demand feature X. But if you approach in good faith and avoid a prideful demeanor, they’re happy to listen. I’ve had have a few success stories in this area already, and by this time next year I want to have a lot more.I’ll be giving a talk at LXJS with this title, by the way. Stay tuned.
I want to end on a hopeful note. It’s easy to think about all these cool things that are coming, and then get depressed about having to support IE8 or Android 2.3 at your job. But that’s the price we pay for an open, interoperable web; we can’t just march to the tune of a single vendor, but instead need to work through this collaborative, cooperative process to build our shared language. In the end, the future is longer than the past, and I look forward to not only living in that future, but in helping to shape it, together with you all.