Account switching is one of the less fun parts of modern computing -- for a while I had registered giveupandusemultiplebrowsers.com. Even when software tries to accomodate these scenarios, the heuristics for when to switch can be tricky, and in some cases require the user to maintain a complex mental model of what state they're in and where they're trying to get to.
Tailscale did not distinguish itself in this regard -- fast user switching was one of our most requested features for the last couple of years. Besides being generally a good thing to do, user switching also plays into the business model – if enthusisast users bring Tailscale to work, they're going to want to be able to quickly switch between their personal and work accounts.
With version 1.34 (released today), we now have the first step: fast-user switching support. Maisem did most of the hard parts (making the Tailscale backend profile-aware, data migration, and untangling what should be global vs. per-account state). I worked mostly on the Mac side, implementing Alessandro's designs but also modernizing our use of the Keychain (where account data is persisted).
I spent a lot of quality time with WWDC sessions from 2010, trying to make custom NSMenuItem views behave just like the real thing, even when using keyboard navigation or VoiceOver. The ironic thing is that Apple itself appears to have given up on this. On modern macOS versions the Wi-Fi and related "menu" controls are actually custom (SwiftUI?) views that don't behave quite like real menus (e.g. they don't highlight items with menu color).
While I was in this part of the codebase, I also cleaned up how the Mac app handles customcoordination servers. Though not officially supported, it's now possible to have profiles on different servers by using the debug menu (finding out how to trigger that is left as an exercise for the reader).
Almost all of the development was in the open-source repo, and started to land a few months ago. It was rather funny to have WebVM notice and start using it a few weeks ago, sort of scooping our own announcement. Their use-case (as a way to get a browser-based VM online) is something I've considered for Infinite Mac.
Brad and I did sneak in an easter egg. I spent a lot of time studying the product spec, and landed it with the commit message "add alternate progress display for SSH sessions". It was very satisfying to have it be discovered without prompting.
Cook reveals him and Jobs disagreed over launching the iPhone as subsidized by the carrier or via receiving a carrier revenue share. He says this was their biggest debate. Jobs, of course, won with the original iPhone launching at $599. Cook got his wish the following year
In retrospect it’s clear that Tim was right, but it would be interesting to know how the original iPhone sales were going — that (presumably) was one of the data points that Tim could use to get Jobs and Apple to change strategies. As it happens I accidentally stumbled on a proxy for such data in that time period, though I didn’t fully grasp its significance until later.
In early 2008 I was browsing around internal the set of internal Google Maps dashboards when I came across one for mobile clients. Google provided the maps data for the iPhone (through 2012’s iOS 6 release), and presumably the iPhone accessed this via the same API that Google's own apps used. By configuring the dashboard to display just new device IDs, it was possible to approximate iPhone sales (assuming that most users tried out the Maps app soon after purchase). Here’s my recollection of what that dashboard showed:
The downward trend after the initial launch was only reversed with the price cut, but even that did not result in a significantly higher plateau. It was also surprising that Christmas was a bigger spike than the initial launch. I also would have thought that the iPod touch spike would be higher (as an iPod touch user myself at the time, I was very excited to get access to Mail and Maps), but perhaps the $20 fee turned off more people.
The dashboard also shows that it was a different era in the Apple/Google relationship — Google had basically real-time visibility into how the iPhone was selling (as I recall the dashboard data was later locked down).
I’ve added a Cloudflare Durable Object-based LAN mode to Infinite Mac, allowing networked games of Marathon, Bolo and anything else that works over AppleTalk. To try it out, use demo.system7.app (or any other subdomain — it defines the “zone” where packets are broadcast and thus instances are visible to each other).
Remember to aim for the ground when using rocket launchers
Though a computer without internet access can feel like a useless brick nowadays, in the 80s and early 90s that was the default state. But even without the internet, local networking was somewhat common, especially in offices (some solutions were more successful than others, and some were elegant kludges). Classic Macs were a part of this, with AppleTalk arriving only one year after the launch of the Mac. Beyond the office use-cases like file sharing and networked printers, this was used for games, with Bolo and Minotaur (Bungie’s first game) being early examples.
After Infinite Mac was announced it took less than an hour for someone to ask for (Marathon) network play — I was not the only one with fond memories of Thunderdome. The Basilisk II architecture is quite modular, and adding networking support for a platform is mostly a matter of implementing a few functions called by the virtual Ethernet driver. There was also an existing option to relay them over UDP, which had pointers for the special broadcast addresses that needed to be handled. oldweb.today had used this approach to get the emulator on the internet.
I began by implementing an ether_js.cpp version of the Ethernet functions that sent/received data from the JS side, and a basic framework for passing the packets to/from the worker to the UI process. Initially that used a BroadcastChannel for local testing, but then I added a Cloudflare Durable Object-based transport (somewhat inspired by this Doom port). That turned into a bit of a yak shave because the tooling and recommended setup for Cloudflare Workers had changed since I last updated them. However, it worked! The appeal of this approach is that it should have reasonable latency since the durable object will be created close to the clients that end up using it.
As I was developing this, I noticed that the emulated Mac would pause for 5 seconds during the boot whenever AppleTalk was enabled. I verified this on actual hardware and confirmed that it was not an emulation glitch. I decided to add some logging to understand why this was happening, and was amused to see that GitHub Copilot was capable of generating suggestions for TypeScript code that parsed AARP packets, which is surely not a common combination:
Unfortunately, after some quality time with Inside AppleTalk it turned out that this delay was by design. AppleTalk nodes will self-assign an address, send out broadcast packets with it, and then wait to see if any nodes report conflicts. While this might be fixable by patching the ROM (a technique that Basilisk II makes heavy use of), it’s not something that I’m doing at this time. Because of the delay in booting, AppleTalk is not enabled in the default Infinite Mac instance, only when using a subdomain (to choose a “zone”). This also has the benefit of ensuring that zones do not get too big. Coincidentally wildcard subdomains recently became free on Cloudflare, enabling this approach.
I added some basic end-to-end latency tracking and was surprised to see that even when using the BroadcastChannel-based transport it was averaging around 8ms, and sometimes was approaching 16ms. These times were suspiciously close to the 60 Hz screen refresh interrupt, and it turned out to be due to my approach of hooking into the input loop — it was only triggered before refreshing the screen. I fixed this by moving input reading to being done on a higher frequency (1,000 Hz) — this both helped with reduced network latency and made mouse input feel smoother as well.
More acceptable ping times
The actual experience when playing Marathon is mixed. The network protocol was designed for LAN play, and does not handle the increased latency of being played over the internet well. If playing for more than 15-20 minutes the game state gets out of sync between players. That being said, it’s satisfying that it works at all.
Infinite Mac has been quite a whirlwind. I wasn’t sure if it would reach the Mac (or retro computing) community, but itdid, and went beyond that too, including to Ars Technica and Hacker News (twice). The most gratifying thing was seeing (and in some cases hearing from) people who were active Mac developers and community members in the 90s, including Andrew Welch, the author of Shufflepuck Cafe, Jorg Brown, James Thompson, and others. It was also great to see people actually usingtheappstomakethings — there’s more to old OSes than just playing games (though those are great too).
Cloudflare Workers are pretty nice for handling traffic spikes
Besides the ego boost, I also got a lot of feedback, and have have made a bunch of changes since then. In addition to addingafewmorethings to the library and making bugfixes, the notable changes are:
I added kanjitalk7.app as a companion site to macos8.app and system7.app. Besides making a new base system image, I also changed how the library is stored, keeping it in a separate disk from the OS. This makes uploading and hosting of alternate OSes much easier, since the same library disk image can be used, instead of duplicating ~1GB of data. I probably could have leaned in more on the content hashing by forcibly aligning files to chunk offsets, but that would have been brittle.
The HFS file system that I was generating had some malformed data structures which became more apparent as the number of files grew (and various B*-tree structures overflowed). After spending a lot of quality time with Inside Macintosh: Files I was able to make twofixes to the machs library and get things working.
The file system also lacked a populated desktop database, which mean that double-clicking on files from other apps did not consistently work. Unfortunately that file format was never reverse engineered, so the easiest way to get it created was to temporarily boot the image, force a desktop DB rebuild, and then persist the results of that.
A lot of CD-ROM games are archived as disk images (especially in the Toast format), so I special-cased the dragging in of those files to instead directly mount them as disks. This unblocks playing of games like Myst.
At some point I was reminded of GUI Central, which was my pre-Mscape Software hobby project (it has a brief mention in my first post). It was a site cataloging Mac customizations (think Kaleidoscope schemes, desktop patterns, etc.). Its main gimmick was that it replicated the 1997-era Mac OS UI in the browser (complete with theme/scheme changing). I have in some ways come full circle.
I’ve extended James Friend’s in-browser Basilisk II port to create a full-featured classic 68K Mac in your browser. You can see it in action at system7.app or macos8.app. For a taste, see also this screencast:
Backstory
It’s a golden age of emulation. Between increasing CPU power, WebAssembly, and retrocomputing being so popular The New York Times is covering it, it’s never been easier to relive your 80s/90s/2000s nostalgia. Projects like v86 make it easy to run your chosen old operating system in the browser. My heritage being of the classicMac line, I was curious what the easiest to use emulation option was in the modern era. I had earlier experimented with Basilisk II, which worked well enough, but it was rather annoying to set up, as far as gathering a ROM, a boot image, messing with configuration files, etc. As far as I could tell, that was still the state of the art, at least if you were targeting late era 68K Mac emulation.
Some research into browser-based alternatives uncovered a few options:
The Macintosh Repository is experimenting with in-browser emulation via either Basilisk II or vMac
However, none of these setups replicated the true feel of using a computer in the 90s. They’re great for quickly launching a single program and playing around with it, but they don’t have any persistence, way of getting data in or out of it, or running multiple programs at once. macintosh.js comes closest to that — it packages James’s Basilisk II port with a large (~600MB) disk image and provides a way of sharing files with the host. However, it’s an Electron app, and it feels wrong to download a ~250MB binary and dedicate 1 CPU core to running something that was meant to be in a browser.
I wondered what it would take to extend the Basilisk II support to have a macintosh.js-like experience in the browser, and ideally go beyond it.
Streaming Storage and Startup Time
The first thing that I looked into was reducing the time spent downloading the disk image that the emulator uses. There was some low-hanging fruit, like actually compressing it (ideally with Brotli), and dropping some unused data from it. However, it seemed like this goal was fundamentally incompatible with the other goal of putting as much software as possible onto it — the more software there was, the bigger the required disk image.
At this point I switched my approach to downloading pieces of the disk image on demand, instead of all upfront. After some falsestarts, I settled on an approach where the disk image is broken up into fixed-size content-addressed 256K chunks. Filesystem requests from Emscripten are intercepted, and when they involve a chunk that has not been loaded yet, they are sent off to a service worker who will load the chunk over the network. Manually chunking (as opposed to HTTP range requests) allows each chunk to be Brotli-compressed (ranges technically support compression too, but it’s lacking in the real world). Using content addressing makes the large number of identical chunks from the empty portion of the disk map to the same URL. There is also basic prefetching support, so that sequential reads are less likely to be blocked on the network.
Along with some old fashioned web optimizations, this makes the emulator show the Mac’s boot screen in a second, and be fully booted in 3 seconds, even with a cold HTTP cache.
Building Disk Images, or Docker 1995-style
I wanted to have a sustainable and repeatable way of building a disk image with lots of Mac software installed. While I could just boot the native version of Basilisk II and manually copy things over, if I made any mistakes, or wanted to repeat the process with a different base OS, I would have to repeat everything, which would be tedious and error-prone. What I effectively wanted was a Dockerfile I could use to build a disk image out of a base OS and a set of programs. Though I didn’t go quite that far, I did end up something that is quite flexible:
A bare OS image is parsed using machfs (which can read and write the HFS disk format)
Software that’s been preserved by the Internet Archive as disk images can be copied into it, by reading those images with machfs and merging them in
Software that’s available as Stuffit archives or similar is decompressed with the unar and lsar utilities from XADMaster and copied into the image (the Macintosh Garden is a good source for these archives).
Software that’s only available as installers is installed by hand, and then the results of that are extracted into a zip file that can be also copied into the image.
(I later discovered Pimp My Plus, which uses a similar approach, including the use of the machfs library)
I wanted to have a full-fidelity approach to the disk image creation, so I had to extend both machfs and XADMaster to preserve and copy Finder metadata like icon positions and timestamps. There was definitely some cognitive dissonance in dealing with late 80s structures in Python 3 and TypeScript.
Interacting With The Outside World
Basilisk II supports mounting a directory from the “host” into the Mac (via the ExtFS module). In this case the host is the pseudo-POSIX file system that Emscripten creates, which has an API. It thus seemed possible to handle files being dragged into the emulator by reading them on the browser side and sending the contents over to the worker where the emulator runs, and creating them in a “Downloads” folder. That worked out well, especially once I switched a custom lazy file implementation and fixed encoding issues.
To get files out, the reverse process can be used, where files in a special “Uploads” folder are watched, and when new ones appear, the contents are sent to the browser (as a single zip file in the case of directories).
Persistence
While Emscripten has an IDBFS mode where changes to the filesystem are persisted via IndexedDB, it’s not a good fit for the emulator, since it relies on there being an event loop, which is not the case in the emulator worker. Instead I used an approach similar to uploading to send the contents of a third ExtFS “Saved” directory, which can then be persisted using IndexedDB on the browser side.
Performance
The emulator using 100% of the CPU seems like a fundamental limitation — it’s simulating another CPU, and there’s always another instruction for it to run. However, Basilisk II is working at a slightly higher-level, and it knows when the Mac is idle (and waiting for the user input), and allows the host to intercept this and yield execution. I made that work in the browser-based version by using Atomics to wait until either there was user input or a screen refresh was required, which dropped CPU utilization significantly. A previous blog post has more details, including the hoops required to get it working in Safari (which are thankfully not required with Safari 15.2).
The bulk of the remaining time was spent updating the screen, so I madesomeoptimizations there to do less per-pixel manipulation, avoid some copies altogether, and not send the screen contents when they haven’t changed since the last frame.
The outcome of all this is that the emulator idles at ~13% of the CPU, which makes it much less disruptive to be left in the background.
With version 1.34 (released today), we now have the first step: fast-user switching support. Maisem did most of the hard parts (making the Tailscale backend profile-aware, data migration, and untangling what should be global vs. per-account state). I worked mostly on the Mac side, implementing Alessandro's designs but also modernizing our use of the Keychain (where account data is persisted).
Cloudflare Workers are pretty nice for handling traffic spikes