Google Reader and Closure Tools #

Since Google Reader makes heavy use of the recently-open sourced Closure Tools, Louis Gray asked me to give a client's perspective on using them. He wrote up a great post summarizing my thoughts, and if you'd like to see the raw input, I've included it below:


There are three pieces to the Closure Tools, the compiler, the library and the template system. They appeared gradually, roughly in that order. The compiler in its current incarnation dates back to Gmail in 2004 (which is why Paul Buchheit refers to it as the "Gmail JavaScript compiler"), with the library and the template system starting a couple of years later.

Reader development started in early 2005, which meant that we always had the compiler available to us, and so except for early prototypes, we always ran our code through it. Until the last month leading up to the Reader launch in October 2005, the size benefits of the compiler were less important, since we were less focused on download time (and performance in general) and more on getting basic functionality up and running. Instead, the extra checks that the compiler does (e.g. if a function is called with the wrong number of parameters, typos in variable names) made it easier to catch errors much earlier. We have set up our development mode for Reader so that when the browser is refreshed, the JavaScript is recompiled on the server and is used with the page when it is reloaded. This results in a tight development loop that makes it possible to catch JavaScript errors as early as possible.

Since Reader development started before the library and template tools were available, we had homegrown code for doing both. There was actually shared code that did some of the same things as basic library functionality (e.g. a wrapper around getting the size of the window, handling different browser versions and quirks). However, that shared code was of various vintages (copied from project to project) and therefore not very consistent in style or quality. Erik Arvidsson's post talks a bit more about the inception of the Closure library (he's one of the co-creators, along with Dan Pupius).

Reader began using the Closure library and template systems gradually, first for new code and then replacing usages of the old shared library and our homegrown code. It was a gradual process, though I tried to keep it organized by doing an audit of all the usages of old code and their Closure equivalents, so that work could more easily be divided up (this was handled during "fixit" periods, where we focus on code quality more than features).

The benefits of the compiler system are tremendous. The most obvious are the size ones, without it Reader's JavaScript would be 2 megaytes, with it it goes down to 513K, and 184K with gzip (the compiler's output is actually optimized for gzip, since nearly all browsers support it). However, all of the above-mentioned checks, as well as many more that have been added over the past few years (especially type annotations) make it much more manageable to have a large JavaScript codebase that doesn't get out of control as it ages and accumulates features.

The library means that Reader is much less concerned about browser differences, since it tries very hard to hide all those away. Over time, the library has also moved up the UI "stack", going from just basic low level code (e.g. for handling events) to doing UI widgets. This means that it's not a lot of work to do auto-complete widgets, menus, buttons, dialogs, drag-and-drop, etc. in Reader.

One thing to keep in mind is that, as mentioned in the announcement blog post, these tools all started out as 20% projects, and for the most part are still dependent on it. If one project needs a feature from the compiler or the library that doesn't exist, they're encouraged to contribute it, so that other teams can benefit too. To give a specific example, Reader had some home-grown code for locating elements by class name and tag name (a much more rigid and simplified version of the flexible CSS selector-based queries that you can do with jQuery or with the Dojo-based goog.dom.query). As part of the process of "porting" to the Closure library, we realized that though there was an equivalent library function, goog.dom.getElementsByTagNameAndClass, it didn't use some of the more recent browser APIs that could it make it much faster (e.g. getElementsByClassName and the W3C Selector API). Therefore we not only switched Reader's code to use the Closure version, but we also incorporated those new API calls in it. This ended up making all other apps faster; it was very nice to get a message from Dan Pupius saying that the change had shaved off a noticeable amount of time in a common Gmail operation.

You can tell that there's something special about this when you look at the ex-Googlers cheering its release. If it had been some proprietary antiquated system that they had all been forced to use, they wouldn't have been so excited that it was out in the open now :)

If you'd like to know more about Closure, I recommend keeping an eye on Michael Bolin's blog. He already has a few posts about what makes it special, and I'm sure there are more coming.

5 Comments

Hi
Really interesting post.
I'm interested to know why the reader team doesn't use GWT for developing Reader? Does raw JavaScript provide some advantage over GWT?
Thanks
Bump to Chris M's question. I'm very interested in GWT, but is it even mainstream at Google yet? Wave seems to use it, but with its current performance, that's not a glowing recommendation.
I definitely can't speak for other teams at Google, but GWT wasn't an option when Reader development started, and once you accumulate enough code, a rewrite isn't appealing.

More generally, the tools/languages are not what has made the biggest difference (or given us the most trouble; we spend a lot more time figuring out what to build and how to bring the concepts of Reader to more mainstream audiences.
I think it is great that the google team is so open with many of the tools it produces. It is interesting the amount of independent invention that happens in the IT world. It seems that many of the tools developed in google just as they were getting their starts in open source land.
Compiler => Various Minifiers
Library => jQuery, Prototype
Template => Trimpath

People face the same problems and smart engineers will often come up with similar but brilliant ways to solve those problems.
@Rob Khor

Comparing the compiler to a minifier igngores one of the compiler's coolest features: adding type safety to javascript through annotations in code comments.

Comparing the templating system to Trimpath ignores the fact that google's templates can be used both client and server side. I'm working on a one page website where everything is rendered with js, so no refresh occurs when going to other pages. However, if the client has js disabled, the templates are put together on the server, reverting to a separate url per page.

Post a Comment