Biz & IT —

How a new HTML element will make the Web faster

Today, the average size of a webpage is 1.7MB—1MB from images—but change is afoot.

Welcome to the standards jungle (we’ve got fun and games)

It is one thing to decide a new HTML element is needed. It's quite another thing to actually navigate the stratified, labyrinthine world of Web standards—especially if no one on your team has ever done such a thing.

Perhaps the best thing about being naive, though, is that you tend to plow forward without the hesitation that attends someone who knows how difficult the road ahead will be. And so the developers working on the picture element took their ideas to the WHATWG, one of two groups that oversees the development of HTML. The WHATWG is made up primarily of browser vendors, which makes it a good place to gauge how likely it is that browsers will ship your ideas.

To paraphrase Tolstoy, every standards body is unhappy in its own way. As Marquis was about to learn, the WHATWG is perhaps most unhappy when people outside it make suggestions about what it ought to do. Suffice to say, Marquis and the rest of the developers involved did not get the WHATWG interested in a new HTML element.

Right around this time, the W3C—which is where the second group that oversees HTML, the HTML WG, is based—launched a new idea called "community groups." Community groups are the W3C's attempt to get outsiders involved in the standards process, a place to propose problems and work on solutions.

After being shot down by the WHATWG, someone suggested that the developers start a community group. The Responsive Images Community Group (RICG) was born.

The Responsive Images Community Group website.
Enlarge / The Responsive Images Community Group website.

The only problem with community groups is that no one in the actual working groups pays any attention to community groups. Or, at least, they didn't in 2011.

Blissfully unaware of this, Marquis and hundreds of other developers hashed out a responsive image solution in the community group.

Much of that effort was thanks to Marcos Caceres, now at Mozilla. Unlike the rest of the group members, Caceres had some experience with writing Web standards. That experience allowed him to span the divide between two worlds—Web development and standards development. Caceres organized the RICG's efforts and helped the group produce the kind of use cases and tests that standards bodies are looking for. As Marquis puts it, "Marcos saw us flailing around in IRC and helped get everything organized."

"I tried to herd all the cats," Caceres jokes. And herd he did. He set up the Github repos to get everything in one place, set up a space for the responsive images site, and helped bring everything together into the first use cases document. "This played a really critical role for me and for the community," says Caceres. "It forced us to articulate what the actual problem was... and to set priorities."

After months of effort, the RICG brought its ideas to the WHATWG IRC. This also did not go well. As Caceres puts it, "standards bodies like to say 'oh, we want a lot of input for developers,' but then when developers come it ends in tears. Or it used to."

If you read the WHATWG IRC logs from that time, you'll see that the WHATWG members fall into a classic "not invented here" trap. Not only did they reject the input from developers, they turned around and, without considering the RICG's work at all, proposed their own solution. It was something called set, an attribute that solved only one of the many use cases Marquis and company had already identified.

Developers were, understandably, miffed.

With developers pushing Picture, and browser makers and standards bodies favoring the far more limited and very confusing (albeit still useful) set proposal, since renamed srcset, it looked like nothing would ever actually come of the RICG's work.

As Paul Irish put it in the WHATWG IRC channel, "[Marquis] corralled and led a group of the best mobile Web developers, created a CG, isolated a solution (from many), fought for and won consensus within the group, wrote a draft spec, and proposed it. Basically he's done the thing standards folks really want 'authors' to do. Which is why this feels so defeating."

Irish was not alone. The developer outcry surrounding the WHATWG's counter proposal was quite vocal, vocal enough that some entirely new proposals surfaced. But browser makers failed to agree on anything. Mozilla killed the WHATWG's idea of srcset on img. And Chrome refused to implement Picture as it was defined at the time.

If this all sounds like a bad soap opera, well, it was. This process is, believe it or not, how the Web you're using right now gets made.

Invented here

To the credit of the WHATWG, the group eventually overcame their not-invented-here syndrome. Or at least partially overcame it.

Compromises started to happen. The RICG rolled support for many of the ideas inset into their proposal. That wasn't enough to convince the WHATWG, but it got some members working together with the Marquis and the RICG. The WHATWG still didn't like Picture, but they didn't outright reject it anymore, either.

To an outsider, the revision process looks a bit like a game of Ping Pong, except that every time someone hits the ball it changes shape.

The big breakthrough for Picture came from Opera's Simon Pieters and Google's Tab Atkins. They made a simple, but powerful, suggestion—make Picture a wrapper for img. That way there would not be two separate elements for images on the Web (which was rightly considered confusing), but there would still be a new way to control which image the browser displays.

This is exactly the approach used in the final version of the Picture spec.

When the browser encounters a Picture element, it first evaluates any rules that the Web developer might specify. (Opera's developer site has a good article on all the possibilities Picture offers.) Then, after evaluating the various rules, the browser picks the best image based on its own criteria. This is another nice feature since the browser's criteria can include your settings. For example, future browsers might offer an option to stop high-res images from loading over 3G, regardless of what any Picture element on the page might say. Once the browser knows which image is the best choice, it actually loads and displays that image in a good old img element.

This solves two big problems. With the browser prefetching problem, prefetching still works and there's no performance penalty. And for the problem of what to do when the browser doesn't understand picture, now it falls back to whatever is in the img tag.

In the final proposal, what happens is Picture wraps an img tag. If the browser is too old to know what to make of a <picture> element, then it loads the fallback img tag. All the accessibility benefits remain since the alt attribute is still on the img element.

Everyone is happy, and the Web wins.

Nice theory, but show me the browser

The Web only wins if browsers actually support a proposed standard. And at this time last year, no browser on the Web supported Picture.

While Firefox and Chrome both committed to supporting it, it might be years before it became a priority for either. Picture was little more than a nice theory.

Enter Yoav Weiss, a rare developer who spans the worlds of Web development and C++ development. Weiss was an independent contractor who wanted Picture to become a part of the Web. Weiss knew C++, the language most browsers are written in, but he never worked on a Web browser before.

Still, like Caceres, Weiss was able to bridge a gap, in this case the world of Web developers and C++ developers. He was in a unique position to be able to know what Picture needed to do and how to make it happen. After talking it over with other Chromium developers, Weiss started hacking on Blink, the rendering engine that powers Google's Chrome browser.

Implementing Picture was no small task. "Getting Picture into Blink required some infrastructure that wasn't there," says Weiss. "I had two options: either wait for the infrastructure to happen naturally over the course of the next two years, or make it happen myself."

Weiss—who, incidentally, has three young children and presumably not much in the way of free time—quickly realized that working nights and weekends wasn't going to cut it. Weiss needed to turn his work on Picture into a contract job. So he, Marquis, and others involved in the community group set up a crowd funding campaign on Indiegogo.

On the face of it, it sounds like a doomed proposition. Why would developers fund a feature that will ultimately end up in a Web browser they otherwise have no control over? But something amazing happened. The campaign didn't just meet its goal, it went way over it. Web developers wanted Picture bad enough to spend their money on the cause.

It could have been the T-shirts. It could have been the novelty of it. Or it could have been that Web developers saw how important a solution to the image problem was in a way that the browser makers and standards bodies didn't. Most likely it was some combination of all these and more.

Enough money was raised to not only implement Picture in Blink, but to also port Weiss' work back to WebKit so WebKit browsers (including Apple's iOS version of Safari) can use it as well. At the same time, Caceres started work at Mozilla and helped drive Firefox's support for Picture.

As of today, the Picture element will be available in Chrome and Firefox by the end of the year. It's available now in Chrome's dev channel and Firefox 34+ (in Firefox you'll need to enable it in about:config). Here's a test page showing the new Picture element in action.

Opera, also based on Blink, will support Picture in the near future. Apple appears to be adding support to Safari through the backport to WebKit, though it wasn't finished in time for the upcoming Safari 8. Microsoft has likewise been supportive and is considering Picture for the next release of IE.

The future of the Web

The story of the Picture element isn't just an interesting tale of Web developers working together to make the Web a better place. It's also a glimpse at the future. The separation between those who build the Web and those who create Web standards is disappearing. The W3C's community groups are growing, and sites like Move the Web Forward aim to help bridge the gap between developer ideas and standards bodies.

There's even a site devoted to what it calls "specifiction"—giving Web developers a place to suggest tools they need, discuss possible solutions, and then find the relevant W3C working group to make it happen.

Picture may be almost finished, but the RICG isn't going away. In fact, it's renaming itself and taking on a new project—Element Queries. Coming soon to a browser near you.

Channel Ars Technica