Infinite Mac 7.5 Weeks Later #

Infinite Mac has been quite a whirlwind. I wasn’t sure if it would reach the Mac (or retro computing) community, but it did, and went beyond that too, including to Ars Technica and Hacker News (twice). The most gratifying thing was seeing (and in some cases hearing from) people who were active Mac developers and community members in the 90s, including Andrew Welch, the author of Shufflepuck Cafe, Jorg Brown, James Thompson, and others. It was also great to see people actually using the apps to make things — there’s more to old OSes than just playing games (though those are great too).

Infinite Mac Requests GraphCloudflare Workers are pretty nice for handling traffic spikes

Besides the ego boost, I also got a lot of feedback, and have have made a bunch of changes since then. In addition to adding a few more things to the library and making bug fixes, the notable changes are:

  • I added kanjitalk7.app as a companion site to macos8.app and system7.app. Besides making a new base system image, I also changed how the library is stored, keeping it in a separate disk from the OS. This makes uploading and hosting of alternate OSes much easier, since the same library disk image can be used, instead of duplicating ~1GB of data. I probably could have leaned in more on the content hashing by forcibly aligning files to chunk offsets, but that would have been brittle.
  • The HFS file system that I was generating had some malformed data structures which became more apparent as the number of files grew (and various B*-tree structures overflowed). After spending a lot of quality time with Inside Macintosh: Files I was able to make two fixes to the machs library and get things working.
  • The file system also lacked a populated desktop database, which mean that double-clicking on files from other apps did not consistently work. Unfortunately that file format was never reverse engineered, so the easiest way to get it created was to temporarily boot the image, force a desktop DB rebuild, and then persist the results of that.
  • A lot CD-ROM games are archived as disk images (especially in the Toast format), so I special-cased the dragging in of those files to instead directly mount them as disks. This unblocks playing of games like Myst.

At some point I was reminded of GUI Central, which was my pre-Mscape Software hobby project (it has a brief mention in my first post). It was a site cataloging Mac customizations (think Kaleidoscope schemes, desktop patterns, etc.). Its main gimmick was that it replicated the 1997-era Mac OS UI in the browser (complete with theme/scheme changing). I have in some ways come full circle.

Infinite Mac: An Instant-Booting Quadra in Your Browser #

tl;dr

I’ve extended James Friend’s in-browser Basilisk II port to create a full-featured classic 68K Mac in your browser. You can see it in action at system7.app or macos8.app. For a taste, see also this screencast:

Backstory

It’s a golden age of emulation. Between increasing CPU power, WebAssembly, and retrocomputing being so popular The New York Times is covering it, it’s never been easier to relive your 80s/90s/2000s nostalgia. Projects like v86 make it easy to run your chosen old operating system in the browser. My heritage being of the classic Mac line, I was curious what the easiest to use emulation option was in the modern era. I had earlier experimented with Basilisk II, which worked well enough, but it was rather annoying to set up, as far as gathering a ROM, a boot image, messing with configuration files, etc. As far as I could tell, that was still the state of the art, at least if you were targeting late era 68K Mac emulation.

Some research into browser-based alternatives uncovered a few options:

However, none of these setups replicated the true feel of using a computer in the 90s. They’re great for quickly launching a single program and playing around with it, but they don’t have any persistence, way of getting data in or out of it, or running multiple programs at once. macintosh.js comes closest to that — it packages James’s Basilisk II port with a large (~600MB) disk image and provides a way of sharing files with the host. However, it’s an Electron app, and it feels wrong to download a ~250MB binary and dedicate 1 CPU core to running something that was meant to be in a browser.

I wondered what it would take to extend the Basilisk II support to have a macintosh.js-like experience in the browser, and ideally go beyond it.

Streaming Storage and Startup Time

The first thing that I looked into was reducing the time spent downloading the disk image that the emulator uses. There was some low-hanging fruit, like actually compressing it (ideally with Brotli), and dropping some unused data from it. However, it seemed like this goal was fundamentally incompatible with the other goal of putting as much software as possible onto it — the more software there was, the bigger the required disk image.

At this point I switched my approach to downloading pieces of the disk image on demand, instead of all upfront. After some false starts, I settled on an approach where the disk image is broken up into fixed-size content-addressed 256K chunks. Filesystem requests from Emscripten are intercepted, and when they involve a chunk that has not been loaded yet, they are sent off to a service worker who will load the chunk over the network. Manually chunking (as opposed to HTTP range requests) allows each chunk to be Brotli-compressed (ranges technically support compression too, but it’s lacking in the real world). Using content addressing makes the large number of identical chunks from the empty portion of the disk map to the same URL. There is also basic prefetching support, so that sequential reads are less likely to be blocked on the network.

Along with some old fashioned web optimizations, this makes the emulator show the Mac’s boot screen in a second, and be fully booted in 3 seconds, even with a cold HTTP cache.

Building Disk Images, or Docker 1995-style

I wanted to have a sustainable and repeatable way of building a disk image with lots of Mac software installed. While I could just boot the native version of Basilisk II and manually copy things over, if I made any mistakes, or wanted to repeat the process with a different base OS, I would have to repeat everything, which would be tedious and error-prone. What I effectively wanted was a Dockerfile I could use to build a disk image out of a base OS and a set of programs. Though I didn’t go quite that far, I did end up something that is quite flexible:

  1. A bare OS image is parsed using machfs (which can read and write the HFS disk format)
  2. Software that’s been preserved by the Internet Archive as disk images can be copied into it, by reading those images with machfs and merging them in
  3. Software that’s available as Stuffit archives or similar is decompressed with the unar and lsar utilities from XADMaster and copied into the image (the Macintosh Garden is a good source for these archives).
  4. Software that’s only available as installers is installed by hand, and then the results of that are extracted into a zip file that can be also copied into the image.

(I later discovered Pimp My Plus, which uses a similar approach, including the use of the machfs library)

I wanted to have a full-fidelity approach to the disk image creation, so I had to extend both machfs and XADMaster to preserve and copy Finder metadata like icon positions and timestamps. There was definitely some cognitive dissonance in dealing with late 80s structures in Python 3 and TypeScript.

Interacting With The Outside World

Basilisk II supports mounting a directory from the “host” into the Mac (via the ExtFS module). In this case the host is the pseudo-POSIX file system that Emscripten creates, which has an API. It thus seemed possible to handle files being dragged into the emulator by reading them on the browser side and sending the contents over to the worker where the emulator runs, and creating them in a “Downloads” folder. That worked out well, especially once I switched a custom lazy file implementation and fixed encoding issues.

To get files out, the reverse process can be used, where files in a special “Uploads” folder are watched, and when new ones appear, the contents are sent to the browser (as a single zip file in the case of directories).

Persistence

While Emscripten has an IDBFS mode where changes to the filesystem are persisted via IndexedDB, it’s not a good fit for the emulator, since it relies on there being an event loop, which is not the case in the emulator worker. Instead I used an approach similar to uploading to send the contents of a third ExtFS “Saved” directory, which can then be persisted using IndexedDB on the browser side.

Performance

The emulator using 100% of the CPU seems like a fundamental limitation — it’s simulating another CPU, and there’s always another instruction for it to run. However, Basilisk II is working at a slightly higher-level, and it knows when the Mac is idle (and waiting for the user input), and allows the host to intercept this and yield execution. I made that work in the browser-based version by using Atomics to wait until either there was user input or a screen refresh was required, which dropped CPU utilization significantly. A previous blog post has more details, including the hoops required to get it working in Safari (which are thankfully not required with Safari 15.2).

The bulk of the remaining time was spent updating the screen, so I made some optimizations there to do less per-pixel manipulation, avoid some copies altogether, and not send the screen contents when they haven’t changed since the last frame.

The outcome of all this is that the emulator idles at ~13% of the CPU, which makes it much less disruptive to be left in the background.

Odds and Ends

There were a bunch more polish changes to improve the experience: making it responsive to bigger and smaller screens, handling touch events so that it’s usable on an iPad (though double-taps are still tricky), fixing the scaling to preserve crispness, handling other color modes, better keyboard mapping, and much more.

There is a ton more work to be done, but I figured MARCHintosh was as good a time at any to take a break and share this with the world. Enjoy!

Update: See also the discussion on Ars Technica and Hacker News (take 2). There is also a follow-up blog post with some post-launch details.