This past weekend I took part in the Foursquare Global Hackathon. I used this opportunity to implement an idea that I had while having dinner with Ann and Dan at Sprout:

The premise of the show How I Met Your Mother is that in the year 2030 the narrator (Ted) is telling his kids how he met their mother. He starts the story 25 years earlier (i.e. in 2005), and thus far (after 6 years), we've gotten a lot of hints, but we haven't met the mother yet. However, it's quite apparent that Ted has in fact been at the same venue as the mother several times. In a hypothetical world where Ted and the mother use Foursquare, I thought it would be neat if they could compare checkin histories and see all the near-misses that they had over the years.

Intersquares does exactly that: you can sign in with your Foursquare account, and then once your checkin history is processed, another user can sign in with their account, and you'll both be told where you were together (whether you knew it at the time or not). This can be great for remembering first dates or for finding close calls.

To see it in action, feel free to see if you have run into me anywhere. There's also a screencast that demos the site.

This was my first hackathon, and I feel like it went pretty well (i.e. I managed to finish something). I definitely tried to keep the hack part in mind, as far as getting something together quickly. The code has a few dodgy technical decisions (keep all checkins in one entity property? what could go wrong?). It was also helpful to build on the same stack as Stream Spigot (App Engine, Python, Closure, Django Templates) so that I could lift a lot of utility code and patterns. Next time, I'd like to be a part of a team though: while being a one man band is satisfying (check out that logo), it does tend to limit the scope to toy-like apps such as this one.

The hackathon has prizes, so if Intersquares intrigues you, your vote is appreciated. Definitely check out some of the other entries too. I haven't gone through all of them yet, but so far Near, Magic Muggle Clock and Plan Your Next Trip all seem really neat.

Update on 9/28/2011: Intersquares was a finalist in the hackathon!

As Jonathan has blogged, "What is the hardest bug you've ever tackled?" is an interesting conversation starting point with engineers, one that I often use to start (phone) interviews. I usually rephrase it as "Describe an interesting or difficult bug that you ran into", since "hardest" often causes people to freeze up as they ponder whether the bug they have in mind is actually the hardest. In any case, most bugs become interesting if you ask "why?" enough.

Along these lines, here's a bug that I ran into in mid-2007 while I was working on Google Reader: Soon after a production push, we noticed that some users were complaining that Reader wasn't loading properly when they reloaded the page. Stranger still, others said that it wasn't working properly initially, but after a few reloads it would start working. Checking things in the office revealed similar inconsistent results: Reader would load for some but not for others. For those for whom Reader hadn't loaded successfully, it turned out to be because of a 404 that was returned when trying to load Reader's main JavaScript file.

This happened soon after Gears support was added to Reader, so we initially suspected some interaction with offline support. Perhaps an old version of the HTML was being used by some users, and that contained a link to a version of the JavaScript file that we didn't serve anymore. However, some quick Dremel-ing showed that we had never served the URLs that triggered 404s until the push began. Stranger still, not all requests for those URLs resulted in 404, only about half.

At this point a bit of background about Reader's JavaScript infrastructure is necessary. As previously mentioned, Reader uses the Closure Compiler for processing and minimization of JavaScript. Reader does runtime compilation, since it supports per-user experiments that would make it prohibitive to compile all combinations at build or push time. Instead, when a user requests their JavaScript file, the set of experiments for them is determined, and if we haven't encountered it before, a new variant is compiled and served. JavaScript (and other static resources) are served with a checksum of their contents in the filename. This allows each URL to be served with a far-future cache expiration header, and makes sure that when its content changes users will pick up changes by virtue of having a new URL to fetch.

The JavaScript URL is used in two places, once embedded as a <script src="..."> tag in the HTML, and once when requesting the file itself. The aforementioned compilation and serving steps happen once for each (identical) frontend machine, and some machines had one idea of what the URL should be, and others had a different expectation. Since the frontends are stateless, it was quite likely for users to request the JavaScript from a different one than the one they got the HTML with the URL from. If there was a mismatch, then the 404 would happen. However, if the user reloaded enough times, they would eventually hit a pair of machines that did think the JavaScript URL was the same.

I said the users were getting "seemingly" identical JavaScript, but there was actually a slight difference when doing a diff (which explained the difference in checksums). One variant contained return/^\s*$/.test(str == null ? "" : String(str)) while the other had return/^\s*$/.test((str == null ? "" : String(str))) (note the extra parentheses in the test() argument). The /^\s*$/ regular expression was distinctive enough that it was easy to map this as being the compiled version of the Closure function goog.string.isEmptySafe, which is defined as:

goog.string.isEmptySafe = function(str) {
  return goog.string.isEmpty(goog.string.makeSafe(str));
};

The goog.string.isEmpty and goog.string.makeSafe calls get inlined, hence the presence of the regular expression test and String() directly (note that the implementations may have changed slightly since 2007).

Now that I knew where to look, I began to turn compiler passes off until the output became stable, and it became apparent that the inlining pass itself was responsible. The functions would not be inlined in the same order (i.e. goog.string.isEmpty and then goog.string.makeSafe, or vice-versa), and in one case the the compiler decided to add extra parentheses for safety. Specifically, when inlining the compiler would check to see if the replacement AST node was of lower precedence that the one it was replacing. If it was, a set of parentheses was added to make sure that the meaning was not changed.

The current compiler inlining pass is very different from the one used at that point, but the relevant point here is that the compiler would use a HashSet to keep track of what functions needed to be inlined. The hash set was of Function instances, where Function was a simple class that had a couple of Rhino Node references. Most importantly, it didn't define either equals() or hashCode(), so identity/memory address comparisons and hash code implementations were used.

When actually inlining functions, the compiler pass would iterate through the HashSet, and since the Function instances corresponding to goog.string.isEmpty and goog.string.makeSafe had different addresses depending on the machine, they could be encountered in a different order. The fix was to switch the list of functions to inline to a List (Set semantics were not necessary, especially given that Function instances used identity comparisons so duplicates were not possible).

The inlining compiler pass had used a HashSet for a long time, so I was curious why this only manifested itself then. The explanation turned out to be prosaic: this was the first Reader release where goog.string.isEmptySafe was used, and there were no other places where there were nested inlineable function calls. (This bug happened around the time we switch to JDK6, which changed HashSet internals, but we hadn't actually switched to JDK6 at that point, so it was not involved).

None of this was reproducible when running a frontend locally or in the staging environment, since all those setups have a single frontend instance (they're of very low traffic). In those cases, no matter which version was compiled and which URL was generated, it was guaranteed to be serveable. To prevent the reoccurrence of similar bugs, I added a unit test that compiled Reader's JavaScript locally several times times, and made sure that the output did not change. Though not foolproof, it has caught a couple of other such problems before releases made it out into production.

The main reason why I enjoyed fixing this bug was because it involved non-determinism. However, unlike other non-deterministic bugs that I've been involved in, the triggering conditions were not so mysterious that it took months to solve.

Discovery (i.e., how a user finds apps to install) is an interesting aspect of app stores*. In some ways, discovery is not necessary: a significant appeal of the store is that it catalogs all the apps, so if the user is looking for a todo list or Twitter client, it's pretty obvious what to search for. However, that assumes that the user has a specific need in mind already, and is aware that that class of application exists.

Ads to promote apps are one way to expose users to apps that they hadn't heard of before. More generally, it's interesting to think of other "ambient" mechanisms that piggyback on existing user activities.

Along these lines, I thought I would play around with the Chrome Web Store set of apps. Ideally, if one is browsing a web site that has a corresponding app in the store, a page action icon would appear to indicate this, similarly to feed auto-discovery notification. Conveniently, hosted apps have a urls section in their manifest which indicates which URLs they want to include within the app. This seemed like a pretty good proxy for which URLs the app was "about". I extracted the URL patterns for a bunch of apps, cleaned them up a bit, and used that to implement a Chrome extension (source) which shows the aforementioned page action when visiting pages that match a Chrome Web Store entry.

Once I had that working, it seemed like a straightforward extrapolation to use the history API to also match browser history URLs against app data. When launched the extension shows apps that match history entries, sorted by recency (this is also available via the extension's options page). The fact that the app data lives locally means that all this matching can be done without uploading the history to a server, which is preferable from a privacy perspective.

Installing apps based on visited websites brings up the "aren't web apps just bookmarks?" question. As it turns out, some apps actually show a pretty different UI than the regular website. For example, the New York Times app features the Times Skimmer UI while the Vimeo app uses a "Couch Mode". The other aspect to consider is that bookmarks have several use cases. In addition to being launchers for frequently used sites, bookmarks are also used for gathering collections of items, remembering where to come back later, etc. Special-casing the launcher use case so that it implies "pretty icons on the homepage" may not be such a bad thing, even ignoring the other extra capabilities of apps.

The URL matching approach has its limitations. For example, the Foursquare Maps app doesn't show up for someone who has foursquare.com in their browser history, even though it ostensibly shows Foursquare data. That's because the app uses OAuth to accesses the Foursquare data on the server-side, so it doesn't have foursquare.com URL in its manifest. This sort of limitation could be fixed by allowing an explicit "this app is about this collection of URLs" entry in the manifest, though there are "interesting" implications to allowing an app to associated itself with a website that it doesn't necessarily own. On the plus side, such a mechanism would also allow this approach to be extended to any app store, even non-web app ones.

* "App store" is used generically in this post. Also, these are my idle weekend thoughts, not official Google promulgations.

Pinky: "Gee, Brain, what do you want to do tonight?"
The Brain: "The same thing we do every night, Pinky — try to take over the world!"

My memory is a bit fuzzy, but from what I remember, if the Brain had set his sights slightly lower, he definitely could have taken over a city, or perhaps a small state as the first step in one night, and left the rest of the world to following nights.

Along these lines, I was talking with Dan about why I thought of Stack Overflow/Exchange as being significantly more successful than Quora. I wouldn't be surprised to find out that they have comparable traffic, users, or other metrics. However, from an outsider's perspective, Stack Overflow made fast progress on its initial goal of being a good programming Q&A site. There was never a clear mission accomplished moment, but at this point its success does not feel in doubt. There were follow-on steps, some more successful than others, and a general upward-and-onward feeling.

On the other hand, Quora's goals from the start were outrageous (in a good way): “Imagine a world where I knew everything that I wanted to know, as long as someone else in the world knew it.” I'm sure that having J.J. Abrams give his thoughts on monster/action scenes is a milemarker on that path. However, it's harder to see how far they've come or to feel like the site has a well-functioning foundation/core functionality, since the path is a continuous curve rather than a step function.*

Google might be considered a counter-example to this; from very early on its goal was quite broad and audacious. However, having a steady stream of corpora to add shows definite progress. There is also the matter of perceived goals versus actual internal goals. Thefacebook was long discounted by some as being a site just for college kids, surely even after they set their sights higher. Having others underestimate your ambition (but not too much, lest they ignore you) seems beneficial.

In the end, this probably reflects my personal bias towards the incremental Ben and Jerry's model. Though less exciting, over time it can lead to pretty good results.

* All of this might be a reflection of my being more aware of what Stack Overflow has done over the years via their podcast; Quora is harder to keep up with.

I'm in the process of re-reading The Baroque Cycle and have gotten curious about Newton's time at the Royal Mint. Ideally, I would like something more detailed than the two paragraphs that Wikipedia devotes to this, but shorter than a 300+ page book*. I've had similar experiences in the past: no matter how much The Economist raved about a ~1000 page history of the British Navy, I was never able to commit to actually reading it all the way through. I think this is more than Internet-induced ADD; I manage to read a book every 4-6 weeks, and dedicating a slot to such a unitasker seems wasteful.

I realize that producing a shorter book on the subject may not be any cheaper or less resource/research-intensive than a long book. I would even be willing to pay the same amount for the digested version as I would for the full version. With recent developments like Kindle Singles there also wouldn't be the issue of fixed production/distribution costs that should be amortized by creating a longer book. Though fiction-centric, Charles Stross has a good explanation of why books are the length that they are.

I used to think that abridged editions, CliffsNotes, and the like were an abomination (as far as not getting the experience the author intended) and for lazy people. To some degree I still do; I think ideally these alternate editions should be produced by the same author, or with the author's blessing.

* As it turns out, there is a 128-page 1946 book about Newton's time at the Mint. Perhaps there was less need to pad then?

Update later that day: Based on the endorsement on Buzz I'll give the (modern) Newton book a try. Part of the reason why I was soured on longer non-fiction books was that I tried reading Operation Mincemeat and was put off by the amount of seemingly extraneous background information and cutesy anecdotes. Incidentally, Operation Mincemeat has a brief appearance in Cryptonomicon, another Neal Stephenson book – I promise that I read other authors too.

Continuing a tradition of making tools for family members, I made for my grandmother a simple Chrome extension that opens all the bookmarks in a folder at startup.

The extension code itself is nothing interesting (making the icon probably took longer). However, it does showcase a limitation of the current Chrome extension system. Since this extension needs to run code at startup, it needs a background page. The extension system architecture overview has a few more details, but briefly, background pages (and any other extension pages) end up in their own process. In this particular case, the background page is not needed after startup, but there is no way to indicate that, so the process hangs around indefinitely, wasting a bit of memory. There is a bug filed for this, part of a broader collection of changes that would enable certain classes of extensions to remove the need for (long-lived) background pages.

A recent Daring Fireball article alludes to a iOS 4.3 Mobile Safari vs. UIWebView difference: "[Safari] uses asynchronous multithreading (UIWebView does not)." John Gruber doesn't make it clear what he means by "asynchronous multithreading", and the term itself seems fishy (doesn't threading imply asynchronous behavior? If you're going to block, why bother with threads?).

I tried to trace the source of this, and saw more possibly related references to asynchronous something or others: The Register says: "[UIWebViews] aren't rendered using Apple's newer 'asynchronous mode'. They're saddled with the old 'synchronous mode', which means means they don't quite look as good." Meanwhile, Ars Technica reports: "Developers have also noticed that full-screen Web apps also don't take advantage of MobileSafari's ability to asynchronously load scripts, which can cause some performance issues—particularly for games. The underlying WebKit engine gained this ability late last year, so it's not entirely clear if this issue is a regression, or if it is just new to MobileSafari and hasn't yet been carried over to WebSheet.app."

Based on the Ars Technica article, we would appear to have at last something concrete to test. However, using the <script defer> test case (mentioned in the WebKit blog post on asynchronous loading) with iOS 4.3, not even Mobile Safari actually loads the script asynchronously, (results - with defer support the result should be around 1,000ms). The same thing with a <script async> test case (results).

It's not clear whether Daring Fireball, The Register, or Ars Technica are talking about the same thing. The Register says they talked to an "unnamed developer", and the Ars Technica article came after (and references) The Register one, so it could just be a game of telephone, where they heard "asynchronous" and assumed it was about script loading. If anyone has concrete (technical) details, they would be appreciated. Alternatively, the iOS 4.3 release of WebCore and JavaScriptCore will show up on Apple's Open Source site eventually, and then it might be possible to investigate this behavior directly.