Infinite Mac has supported a limited form of persistence since its initial launch. This was done by exporting the contents of the “Saved” folder in “The Outside World” to IndexedDB when the emulator was shut down. While this worked, it had several limitations:

• Saving was best-effort during the page unload process. The beforeunload event that it relied on does not always fire, and it required an asynchronous postMessage step by which point the process for the page may have been terminated.
• It relied on Basilisk II and SheepShaver’s ExtFS support (which creates a virtual drive using the File System Manager). This meant that it was not available in Mini vMac-based emulators (i.e. anything for System 6 or earlier).
• Even when ExtFS/File System Manager is available, the virtual drive has several limitations: some software thinks it’s a networked drive and refuses to run on it, and it’s not bootable.

With the improved disk abstractions that enabled remote CD-ROM support, I realized that that I could mount another hard disk image in the emulated Mac, intercept read and write calls to it, and handle them from a persisted store. This was made possible by the new-ish origin private file system (OPFS) API, and specifically the ability to do synchronous file operations from a worker (without heroics). It’s supported in all modern browsers, including WebKit-based ones. The synchronous part is key because it allows all changes to be persisted as they happen, instead of during page unload.

The specific approach that I settled on was to have a mostly empty 1GB HFS disk image that serves as a “Saved HD”. Any modifications to it (tracked using the same 256KB chunk granularity that’s used for disk image streaming) are persisted in an OPFS file (a second file maintains the indexes of all modified chunks). The “mostly” empty part is because there is some metadata (desktop database, custom icon) as well as a Read Me file that are in the initial state of the disk image. This system makes the actual OPFS space that’s consumed to be proportional to the amount of data written, instead of ballooning to 1GB from the get go. It could also be extended to support a snapshotting/copy-on-write system down the line.

While OPFS is pretty broadly available, Infinite Mac still has enough visitors from older browsers that I wanted to detect support for it (the API is also not available in private Safari windows). One annoying gotcha is that the synchronous file API is only available in workers, and there’s no way to detect its presence from the main browser process. I therefore have to create a temporary worker to check for the existence of the createSyncAccessHandle and related functionality. The check is done as early as possible and the results are cached, so that most of the time we can synchronously determine the support level.

With all that in place, it’s possible to use the Saved HD as a “real” disk in the emulated Mac. This enables quite a few capabilities:

• You can install system software on Saved HD (either by copying it from an existing disk or via a System Software CD-ROM from the library). It can then be used as a startup disk (if running a custom instance with no other disks), allowing custom extensions and control panels to be installed and persisted across sessions.
• The contents can be exported and imported (from the emulator settings dialog), allowing backups and sharing across browsers and machines.
• The contents can also be exported to a .dsk disk image file, so that they can be used in Basilisk II, SheepShaver and other native emulators.
• The same disk is mounted in all instances, so it's also a way to move data from one to system disk to another.

Your own private System 7 install

The end result is a way to make Infinite Mac feel like “your Mac” while still keeping everything local and fast. That also lines up well with my goal of keeping the site a fun hobby project — while persisting disks on a server would be neat, it would also be a more expensive and complex operation.

Update on October 1, 2023: This has been extended to also support generating of disk device images (.hda files), which allows Infinite Mac prepare disks for BlueSCSI devices. Ron's Computer Videos has a walkthrough of the process.

The first proper blog post on this site was 20 years ago today. It lived under a different URL, I didn’t register persistent.info until later (when .info was a new-and-shiny top-level domain). It also looked different (I don’t have a screenshot of the first version, but I switched themes a few months later) and was published by very different software (Movable Type 2.64), but there is continuity from then to today.

Later that day I imported a bunch of older programming journals, thus the entries go back almost 25 years. The first few years are not really blog posts, but it’s been nice to have snapshots of early work, even if they’re not all gems. I’ve been backfilling a projects page, and the posts also serve as tangible artifacts to link to, even if the project itself is long-dead.

The trend for posts per year is looking encouraging when looking at the past decade, here’s to a few more (while computers still work).

Here are a few highlights of what’s been happening with Infinite Mac since my last update.

Improving Disk Abstractions

My broad goal this time around was to make it easier to load external software. To lay the groundwork for this, I first needed to improve some of the disk abstractions that the project used. One of the things that makes the emulated Macs relatively fast to boot is that they stream in their disk images, since much of it is not needed at startup (e.g. Mac OS 9 only reads 75MB out of a 150MB install). I had implemented this streamed/chunk approach pretty early in the project, when I understood the emulator internals (and Emscripten) less well. It worked by monkey-patching the read/write operations in Emscripten’s MEMFS filesystem (the default when doing an Emscripten build).

Besides being somewhat hacky, this had the downside of requiring that all mounted disks be loaded into memory as ArrayBuffers (hence the name MEMFS). While the system software disks are relatively small, the Infinite HD disk with pre-loaded software is 1 GB, thus this was leading to significant memory use. I had added some basic instrumentation of emulator errors, and running out of memory was surprisingly common (6.5% of emulator starts), presumably due to low-memory iOS devices or 32-bit Chrome OS devices. If I was going to add the ability to load additional (CD-ROM-sized) disk images, the memory use would increase even more.

I briefly explored Emscripten’s other file system options, but they all didn’t seem like quite the right fit. Fundamentally the issue was that all disk access was going through too many layers of abstraction, and the intent was lost. The emulator was operating against the POSIX file system API (open, read, on a file descriptor etc.) and by the time Infinite Mac’s code ran, it required extra bookkeeping to know which file was being requested (it could be a non-disk file), and the choice for the backing store for that file (as an ArrayBuffer) had already been made by Emscripten.

Before

After

While many problems in computer science can be solved with a layer of indirection, in this case it was a matter of removing one or two. The emulators are cross-platform systems themselves, and they have a way of swapping out the POSIX file backing for disks with others for other platforms. By adding more direct “JS” disk implementations in both Basilisk II/SheepShaver (hereafter “Basilisk”) and Mini vMac and then implementing the JS APIs on the Infinite Mac side, the entire POSIX and Emscripten layers could be bypassed. We now only need to use as much RAM as the number of 256K chunks that have been accessed (and there is enough flexibility in the system that a LRU system could also be implemented to keep the working set fixed in size). After this change the out-of-emory error rate went to 0.3%.

The next thing to tackle was avoiding the need to restart the emulators when mounting additional disks. This only affected the Basilisk version, since Mini vMac already had built-in support for mounting disks on demand. I assumed that this was not a fundamental limitation, all the emulators supporting mounting (real) removal media on demand already. It turned out to be a missing feature in Basilisk’s disk abstraction, and once I added that it was possible to mount disks immediately there too.

CD-ROM Library

With this in place, I was able to start working on the CD-ROM “library” feature I had envisioned. This was based on the observation that the Internet Archive has a large collection of disk images in the .iso format, include many for the Mac. Furthermore, the files are accessible without any kind of authentication, and support HTTP range requests. This meant that it’s possible to take the same chunked/streaming approach I was using for my disk images and use it for these files without any kind of pre-processing. It is rather amusing that download speeds and latencies over the Internet are better than those of 1-2x CD-ROM drives from the early 90s (150-300 KB/sec and 100-200ms seek times).

I do these range requests in simple handler in a Cloudflare Worker, mostly so that they end up being same-origin requests. I later found that there is a CORS gateway for the Internet Archive, but it appears to be a secondary service, and doing the fetching myself means that I can benefit from Cloudlare’s edge caching. However, switching to doing direct client-side loading is something to consider if the bandwidth costs become prohibitive (though for now donations are more than covering the costs).

One gotcha that I ran into this that some disk images are compressed. While there ways to make .zip files seekable, they require re-compressing, which would defeat the purpose of making this be a minimal setup/low overhead feature. For now I’ve chosen to skip over compressed files (or support other disk image formats that support compression, such as .dmg).

Continuing with the project’s curation philosophy, I wanted to have a nice browsable UI of CD-ROMs, ready to be mounted in the emulated Mac. I ended with a collection of metadata files that specify disk image URLs and cover art that is processed by a simple importing script. The generated catalog is rendered in a pop-up folder-inspired UI:

I added some of my favorites and got a few more suggestions via Mastodon. Around this same time a pretty throughly researched article on the first CD-ROMs appeared, and I hoped to include the ones referenced too. However, most of the early CD-ROMs were actually just taking advantage of the ability to include audio CD tracks, which is something that I have not gotten to work yet, so I was not able to add them.

I did also add the ability to mount arbitrary CD-ROMs via a dialog, so if you come across something on the Internet Archive that catches your eye, you can give it a try (you can even drag-and-drop the link from another tab). I also made the Infinite HD disk image 2GB (i.e. with 1GB of free space), so that it’s available as a destination for any software that requires installation to a local hard drive. This is another way in which the change to not require disk images to be loaded into RAM paid off.

Custom Instances

Around the time that I was wrapping up CD-ROM support, I came across the recently-launched Classic Macintosh Game Demos site. It’s a “kindred spirit”, in that it’s trying to make things easily accessible and has a curatorial bent. Jules Graybill (the author) was sourcing the game demos from CD-ROMs that came with magazines, images of which were also uploaded to the Internet Archive. I added the ability to specify a CD-ROM URL via a query parameter and reached out to him to see if he wanted to make use of it (to allow the games to be played in-browser). He quickly implemented it, which is a great example of the flexibility of web-based projects — they can be made to work together (some might even say “linked” or “mashed up”) with minimal coordination.

Shortly after, Jules filed an issue asking about an additional system extension he needed included on the system disk. Extrapolating from the ability to load CD-ROMs from URLs, I wondered if the same could be done for the system disk, so that he could provide his own image with exactly the extensions that he needed. And while we’re at it, why not allow the type of machine to be chosen via a URL parameter too, and control over other Infinite Mac features (AppleTalk support or the extra Infinite HD disk). Getting perhaps a bit carried away, I ended up building a Calculator Construction Set equivalent for emulated Macs, available at infinitemac.org/run.

To reduce the cognitive dissonance (and to have a bit of fun), I made the UI resemble the look-and-feel of the OS that is being booted. I had already added some classic- and Platinum-style controls for other bits of configuration UI, but this dialog also required implementing popup menus and checkboxes (the Appearance Sample app from the Appearance SDK was invaluable in getting a view of all “modern” Mac OS controls). There’s still a bit of a an uncanny valley feel, perhaps due to fonts not being the same and modern screens having higher logical DPI (thus everything feeling smaller than it used to), but hopefully it will get closer over time.

You can also use this customization mode to have “Frankenstein” instances. For example, you can load the System 1.0 image while booting with System 7, allowing you to use a modern version of ResEdit to poke around the initial versions of the System and Finder. Or you can use this with any system disks that you may happen to come across (e.g. if you wanted to see what the Slovenian version of System 7.5 looked like).

Odds and Ends

While the goal of the site is to make as many versions of classic Mac OS available in the browser, I realized some time after launch that 40+ releases is a lot to scroll through, and that more notable ones might get lost among all of the point releases. I therefore added a simple filter toggle to make the list that’s initially shown more manageable.

A longstanding issue was that dissolve animations in HyperCard would run very slowly, something that affected the native build of Basilisk II too. It turned out to be due to a missing implementation of high-resolution timers, which was recently fixed on the native side. While simply updating my Basilisk II fork did not get the fix for free, I did now have enough clues to implement my own version of it.

There was the usual round of upgrades for the software stack: a new way to install Emscripten, transitioning from create-react-app to Vite (that one was a yak shave), and switching to the latest version of TypeScript, React, and other dependencies.  Though site performance hasn’t really been an issue, the switch to Vite made the initial bundle size more visible, so I spent a bit of time adding moving some things to dynamic imports and some preloading to pick up some easy wins.

I haven’t entirely decided what I’m going to focus on next, but I’m leaning towards improving the ability to persist changes to disks — it’s a shame to spend time installing software from CD-ROMs and then lose it all when closing the tab. The improved file system abstractions should make it easier to implement some of my ideas here.

I finally got access to the recently-launched Slack canvas feature. This project was the last thing I worked on before I left Quip/Slack/Salesforce in April of 2022, and I was curious how it had evolved since then.

Canvas started as a prototype in mid-2021 to reuse Quip technology to power a collaborative editing surface inside of Slack. The initial phase involved Michael Hahn and I doing unspeakable things with iframes¹ to get the two large codebases to work together² with minimal changes. This allowed a relatively rich feature set to be explored quickly, but it was not something that was designed to be generally available. At the time of my departure the work on productionizing (the second 90% of the project) had just begun.

The first thing that becomes apparent is that the roots of canvas in Quip are entirely hidden from the user — no Quip references in the announcement or anywhere in the UI. This makes sense from a “don’t ship your org chart” perspective — no user should care how a feature is implemented. However, if you peek under the hood, you can start to see the some Quip tidbits. The most obvious place to start is to look for network requests with quip in them — a couple of which happen when loading a Slack canvas:

The “controller” is the core logic of Slack canvas editor, and we if load one of those URLs, we see even more Quip references:

The DOM also resembles Quip’s, down to the same CSS class names being used. The need to scope/namespace them to avoid colliding with Slack’s was one of the open questions when I left, but I guess Slack has a BEM-like structure which ensures that Quip’s simpler class names don’t collide (as long as they don’t integrate another similar un-prefixed codebase). There are also no iframes in sight, which is great.

Quip also had extensive in-product debugging tools, and I was curious if they also made the transition to Slack canvas. They’re normally only enabled for employee accounts, but as hinted there is a way to enable them as a “civilian” user too. A couple of commands in the dev tools, and I was greeted by the green UI that I had spent so many years in:

I was also hoping that copying/pasting content from Quip into Slack canvas was a backdoor way to get some of features that have not (yet?) made the transition (spreadsheets, date mentions, etc.), but it does not appear to work.

On the mobile side, I had explored reusing Quip’s hybrid editing approach in the Slack iOS app, including the core Syncer library. Hooking up Console.app to an iOS device shows that the Syncer (and thus Quip) are still involved whenever a canvas is loaded.

One of the open questions on mobile at the time of my departure was how to render Slack content that’s embedded in a document. Quip’s mobile editor is a (heavily customized) web view, so that we can reuse the same rendering logic on all platforms. It's possible to see that the canvas rendering is still web-based by inspecting the Slack app bundle (Emerge Tools provides a nice online tree view) – there is a mobile_collab.js file which implements document rendering:

Slack on the other hand is an entirely native app. Porting Quip’s editor to native components didn’t seem feasible on any sort of reasonable timeframe. It was also not appealing to reuse Slack’s web components on mobile, since they weren’t designed for that (either from a UI or data loading perspective). I had speculated that we could leave a “placeholder” element in the web view for Slack-provided UI (like a message card), and then overlay the native component on top of it. But I wasn’t sure if it would be feasible, especially when the document is scrolled (and the native view overlay would have to be repositioned continuously).

It’s not as easy to inspect the view hierarchy of an iOS app (without jailbreaking), so I can’t verify this directly, but it would appear that this placeholder/overlay approach was indeed chosen. Most of the time, the native Slack UI is positioned perfectly over the document. However, in some edge cases (e.g when a scroll is triggered because the keyboard is being brought up), things end up slightly out of sync, and the placeholder is visible:

This is my first time being on the outside of a project while significant work on it continued (unlike other times), and it’s been fascinating to observe. I congratulate all the people involved in shipping Slack canvas, and will cheer them on³.

1. I later realized I had done the same thing 15 years earlier, getting Reader iframed into Gmail as another proof-of-concept.
2. At one point we had slack.com iframing quip.com which in turn was iframing slack.com again (so that we could show Slack UI components inside documents), an architecture we took to calling “turducken.”
3. Especially if they bring back syntax highlighting for code blocks.

It doesn't feel like it's been 5 years since my last post about Reader, but I guess the past few years have suffered from time compression. For this anniversary I don't have any cool projects to unveil, but that's OK, because David Pierce wrote a great article – Who killed Google Reader? – that serves as a nice encapsulation of the entire saga.

Other Reader team members and I had a chance to talk to David, and the article captures all of the major moments. Some things ended up being dropped though; there's enough twists and turns (in the social strategy alone) that a a whole book could be written. Here's some more "fun" tidbits from Reader's history:

The article talks about "Fusion" being Reader's original/internal name (and how the "Reader" changed how it was perceived and limited its scope). The reason why "Fusion" was not used was because Google "wanted the name [Fusion] for another product and demanded the team pick another one. That product never launched, and nobody I spoke to could even remember what it was.". Fusion was the initial name that iGoogle launched under, as can be seen in this article from mid-2005 (iGoogle itself was went through some naming changes, changing from Fusion to Google Personalized Homepage before ending up as iGoogle (its codename) in 2007). Finding the breadcrumbs of this story was somewhat difficult because Google later launched a product called Google Fusion Tables (not surprisingly, it was also shut down).

In terms of naming, these were other names that were considered, so "Reader" was as worst-except-for-all-the-rest sort of thing:

• Google Scoop (which team took to referring to as "Scooper", as in pooper scooper)
• Google Viewer
• Google Finder
• Google Post

At one point during the (re-)naming saga Chris put in "Transmogrifier" as a placeholder name, with the logo being one of Calvin's cardboard boxes. During the next UI review Marissa Mayer was not amused (or perhaps it was hard to tell what the logo was in those pre-retina days), and the feedback that we got was "logo: no trash".

A low point in the interal dynamics was hit in 2011. I had made some small tweaks (in my 20% time) to fix an annoying usability regression where links were black (and thus not obviously clickable). Since we were getting a lot of flack for it on Twitter, I tweeted from the Reader account saying that it was fixed. A few hours later, I got a friendly-but-not-really ping from a marketing person saying that I need to run all future tweets by them, since there was an express request from Vic Gundotra to limit all communication about Reader, lest users think that it's still being actively worked on. That was the second-to-last tweet from the official account, the next one was the shutdown announcement.

After Twitter blew up at SXSW 2007 there was a start of a "I don't need Reader/RSS, Twitter does it for me" vibe amongst some of the "influencers" of the time. I posted a somewhat oblique tweet comparing the Google Trends rankings of "google reader" and "twitter" (with "toenails" being a neutral term to set a baseline), showing that Reader dwarfed them all (the graph looks very different nowadays). I couldn't understand why someone would want to replace Reader with a product that had no read state, limited posts to 140 characters, and didn't even linkify URLs, let alone unfurl them. In retrospect this was a case of low-end disruption.

I wrote a valedictory blog post about the instrumentation we're adding to the Tailscale client to get a handle on battery life issues. It's been an interesting "full-stack" project, involving thinking about cell phone internals, hacking on the Go standard library, and exploring visualization options.

Darwin was one of the first things Apple open-sourced (24 years ago). It's been mostly a "throw it over the wall" approach, but being able to peek under the hood has been very handy.

I wrote a short tailscale.dev blog post showing how we've been able to improve Tailscale's macOS and iOS apps by seeing how things are implemented.