Infinite Mac: Turning To The Dark Side #

tl;dr: No, not Windows¹, but I’ve decided to expand Infinite Mac to the the black hardware that Mac users secretly lusted after in the 80s and 90s: NeXT.  There is now a runnable collection of all notable NeXTStep versions, going from the initial 0.8 preview in 1988 to the final OPENSTEP 4.2 release in 1997.

NeXTSTEP 3.3 running DOOM, DoomEd and WorldWideWeb

Porting Previous

About a year ago I came across the Previous emulator – it appeared to be a faithful simulation of the NeXT hardware and thus capable of running NeXTStep. While including it in Infinite Mac would be scope-creep, NeXT’s legacy is in many ways more relevant to today’s macOS than classic Mac OS. It also helped that it’s under active development by its original creator (see the epic thread in the NeXT Computers forums), and thus a modern, living codebase².

Previous is the fifth emulator that I’ve ported to WebAssembly/Emscripten and the Infinite Mac runtime, and it’s gotten easier. As I'm doing this work, I’m developing more and more empathy for those doing Mac game ports – some things are really easy and others become yak shaves due to the unintended consequences of choices made by the original developers. Previous is available on multiple platforms and has good abstractions, so overall it was a pretty pleasant experience.

The main challenge was the pervasive use of the SDL library – I don’t want any of it as a platform abstraction, since I have my own. I chose to replace the entry-point and several subsystems to avoid bringing in too much SDL code. Even with that, I still needed to have some stub files for dependencies that I couldn’t omit altogether. On the other hand, it was a pleasant surprise that adding sound support was trivial — the APIs that it needed mapped 1:1 with the ones I’d already exposed from the Infinite Mac runtime.

With the initial emulator being brought up, there were some more fun tasks, like adding a NeXT-style monitor frame and a NeXT appearance to the Infinite Mac controls (working on them is giving me Kaleidoscope scheme flashbacks).

Infinite Mac custom instance configuration dialog, rendered with a NeXTStep appearance

Once I had Previous running I did some profiling and was discovered the same excessive wasm-to-js/js-to-wasm ping-ponging that I noticed in DingusPPC – it also uses setjmp/longjmp to implement an exception handling mechanism for the MMU. The fix was also very similar: intentionally add an extra wrapping function that contains the CPU emulation loop.

The performance impact of the fix was not as noticeable because Previous tries to accurately simulate the performance characteristics of the original hardware (this is also why boot times are much longer). In fact, I had originally missed the speed checking that the native main loop does, and the emulator was running too fast, leading to timing-dependent things like double-clicking not working.

Speaking of mouse input, Previous uses relative mouse input — it accurately simulates real hardware so all it can do is behave as though a physical mouse is moving and generating updates. This was the case with DingusPPC too, and while the Pointer Lock API makes this work reasonably well on desktop browsers, it’s not a good fit for the mobile web. To make input more accessible on touch-enabled devices, I added a virtual trackpad mode that turns swipes and taps on the screen into mouse movements and clicks. This also helps with double-clicking and making pre-Mac OS 8.0 pull-down menus more usable, though it is somewhat finicky to trigger the drag-lock gesture (I now see why it’s so hard for other touchpad vendors to replicate all of Apple’s gestures).

As part of doing some of the integration work I became aware of just how much of a minority player NeXT was in the 80s and 90s computing world (even more so than Macs). For example, they had their text encoding/character set, and while Python (and other systems) have native support for the contemporary MacRoman encoding, I had to add my own to handle NeXT’s. Similarly, there is a convenient Python library for dealing with HFS disk images, but I could not find any for the UFS variant that NeXT used³.

The disk images with NeXTStep installs are quite large (hundreds of megabytes per version), and the Workers KV-based storage that I had been using was becoming increasingly ill-suited for them (and expensive). As part of this project I also switched to Cloudflare R2 (their S3-like object storage system). On one hand I’m not thrilled with the site becoming dependent on additional services, but on the other I do want to keep costs down. Luckily the dependency is not invasive — it would be trivial to store the disk chunks anywhere else (or to keep storing them as static files, the way I do in the local dev server).

Exploring NeXTStep

Although I was familiar with NeXTStep at a theoretical level and had spent a few hours using a slab in the mid 1990s, this was my first time using it in-depth. My first observation was that having it side-by-side with the contemporary Mac system software makes it even more apparent how much of an advance it was.

Notable OS releases of 1988: System 6.0 and NeXTStep 0.8
System 6.0 and NeXTStep 0.8: two very different releases from 1988

As I was preparing the system images with the various NeXTStep/NexTSTEP/OPENSTEP releases⁴, it was fascinating to see NeXT’s various strategy shifts over the years represented in the canned email from Steve Jobs that was present in the Mail app:
Steve Jobs email from NeXTStep 2.0 Steve Jobs email from NeXTStep 3.0 Steve Jobs email from NeXTSTEP 3.3
Strategy du jour: productivity (NeXTStep 2.0), multimedia and internationalization (NeXTStep 3.0), object-oriented development (NeXTSTEP 3.3)

I was also amused to notice a parallel with the early Mac system releases: the initial release is hard to find in pristine form (a copy of System 0.97 was only tracked down late last year). In both cases it’s it because of the very small number of copies produced, and because they shipped on read/write media (floppy disks for the Mac, magneto-optical disks in NeXT’s case), and thus ended up being used for personal storage too. For NeXTStep 0.8 the archive of images that forum member mikeboss created was the best source that I could find, though I’m happy to switch things over if a better one turns up.

As part of installing all of the OS versions I discovered that starting with NeXTStep 3.0 there is support for the HFS file system used by contemporary Macs. This means the Saved HD drive that has persistent storage also ends up working here. It can even be used as a virtual sneakernet to move data between NeXT and Mac installations on Infinite Mac (which means that it’s possible to get data in and out of the machines via Macs which have “The Outside World“ drive).

To make the systems more than a quick curiosity, I also created a NeXT version of the “Infinite HD” drive with pre-installed software. I had read stories about Doom and WorldWideWeb being developed on NeXTStep, and it was nice to make them accessible. I also added a NeXT version of the CD-ROM library feature, with one-click access to disks that have been preserved at the Internet Archive.

NeXT CD-ROM Library on Infinite Mac

As part of finding more NeXT software to try out, I came across Daydream, which allows Mac software to run on black hardware. It’s technically not an emulator, instead it takes advantage of the fact that NeXT and Macs both used the Motorola 68K CPUs. It effectively turns your NeXT machine into a Mac that you dual-boot via the help of an adapter kernel. The original version needed a hardware dongle with genuine Mac ROMs, but a few years ago the NeXT Computers forum resurrected it such that it can run entirely via software. It’s this variant (known as Darkmatter) that is included on Infinite HD. In some ways this justifies the whole Previous side-quest — it’s just a more convoluted way to run the same classic Mac software as the rest of the Infinite Mac emulators.

Darkmatter running System 7.1 emulated on NeXT hardware emulated in Previous in a browser

Things happen

Christopher Nolan (sort of) knows about Infinite Mac. I did a minor site redesign. After recalling the KanjiTalk New Year’s Day easter egg, I added custom date/time support so can you see it anytime you want. Infinite Mac is being used as a tool to make Playdate game art and other toys. You can now load files with resource forks straight from the macOS Finder. Apeiron now works. And most excitingly, DingusPPC has progressed to point where the Mac OS X 10.2 installer starts up.

Update: See also the discussion on Hacker News.

  1. Also, PCjs already has the Windows niche well-covered
  2. andreas_g is the Previous author, and he was very helpful in the thread I started about my porting work
  3. I did take a stab at writing a HexFiend template for UFS/NeXT partition maps
  4. See these pages for details on the capitalization rules.

Infinite Mac: The Case Of The Missing Text #

I had previously mentioned that I ended up fixing some DingusPPC CPU emulation bugs. This was my first time working at this (lower) level of the Infinite Mac emulators, and I thought it would be interesting to write up such an investigation.

There was an issue that was immediately apparent when I first got DingusPPC building: while starting up from the 7.1.2 Disk Tools floppy would result in a functional-looking Finder, doing the same from the 7.1.2 install CD would end up with many rendering glitches — notably no text, but also missing icons.

System 7 Finder window
7.1.2 Disk Tools Finder
System 7 Finder window with no text or icons drawn
7.1.2 Install CD Finder

Classic System Software is mostly a closed-source black box (outside of leaks), and while DingusPPC is open source, it’s a codebase that is still pretty foreign to me. In the absence of other ideas, I figured that creating a more reduced test case would be a good use of time. This reminded me of my investigations into Chromium/WebKit rendering bugs, where half the battle was reducing a complex web page down to a minimal test case.

I examined the two system folders and observed that the CD-ROM one was quite a bit larger, with additional extensions, a different enabler¹, and a bigger System suitcase.

System Folder from the 7.1.2 Disk Tools floppy, list view System Folder from the 7.1.2 CD, list view

I created a read-write copy of the CD-ROM system folder and used a native build of Basilisk II to operate on it, removing or switching out files one a time to morph it into the floppy version, and looking for any differences when booting it in DingusPPC. It turned out that switching out the “Minimal PowerPC Enabler” on the floppy with the full “PowerPC Enabler” one from the CD-ROM was the key difference — with the full enabler text would not render.

I then opened the two enablers in ResEdit and compared them — the full one had a bunch more resources. I thus repeated the process, deleting extra resources one at a time to morph the full enabler into the minimal one.

ResEdit view of resources in the Minimal PowerPC Enabler ResEdit view of resources in the regular PowerPC Enabler

It turned out that the culprit was the ntrb resources, specifically removing ID 16 (StdTextTraps) and 18 (QuickDrawTextTraps) would restore text rendering. They contain PowerPC executable code — the giveaway is the Joy!peff prefix that is the header of the Preferred Executable Format. “Traps” refers to the system/Toolbox calls. My assumption is that the ROM of the first generation of Power Macintoshes was finalized relatively early, and contained mostly 68K code that would need to be emulated². The PowerPC enabler contained native implementations of more of the Toolbox, which were loaded in via these ntrb resources. These particular resources had native implementations of the text subsystem, which was very much in line with where the buggy behavior was.

Between resource_dasm and some help from DingusPPC Discord member joevt I was able to get a list of the 14 Toolbox traps that were being reimplemented as native PowerPC versions. The one that immediately stood out was StdTxMeas – here’s a description of it from Inside Macintosh: Text:

The QuickDraw text routines use two of the bottleneck routines extensively—one to measure text (StdTxMeas) and one to draw it (StdText). Most of the high-level QuickDraw text routines call the low-level routines. The use of bottleneck routines provides flexibility to QuickDraw and applications that need to alter or augment the basic behavior of QuickDraw.

While I had significantly narrowed where things were going wrong, I was reaching the limits of what I could via basic file system and ResEdit hacking. Ideally I would have a simple test program that just called StdTxMeas, so that I could examine its behavior in isolation.

It was thus time to crack open CodeWarrior and write a simple test program. I used the SillyBalls sample code as a starting point — there certainly used to be a lot of boilerplate in getting a basic window up on the screen. I created a simple test function that draws a string and then its bounding box as determined by StdTxMeas. Lacking the ability to render debugging text, I used FrameRect as the output mechanism.

void TestStdTxMeas(WindowPtr mainWindowPtr) {
    Rect measureRect;
    Str255 testString = "\pHello World";
    Point number = {1, 1};
    Point denom = {1, 1};
    FontInfo fontInfo;
    int width;
    
    SetPort(mainWindowPtr);
    MoveTo(20, 20);
    TextSize(12);
    
    DrawString(testString);
    
    width = StdTxMeas(testString[0], testString + 1, &number, &denom, &fontInfo);
    
    // +1 for the width so that if we get a zero we can still see the rect.
    SetRect(&measureRect, 20, 8, 20 + width + 1, 24);
    FrameRect(&measureRect);
}

I set the program as the startup item (so that I wouldn’t have to navigate blindly in a half-drawn Finder window) and observed its behavior under both emulated and native implementation of the text traps.

QuickDrawTester window showing expected output of "Hello World" surrounded by a rectangle QuickDrawTester window showing incorrect output with just a black line

Sure enough, it looked like when using native PowerPC code, StdTxMeas would end up returning 0 instead of the expected value. I suspected a bug in DingusPPC’s PowerPC CPU emulation, but it was unclear where. I had some false starts:

  1. I tried to disassemble the StdTxMeas implementation – I got lost when going through too many layers of indirection due to the calling convention.
  2. I looked at its 68K implementation and checked some of the intermediate building blocks – the width table looked correct.
  3. I tried to get MacsBug running so that I could single-step through it – I was not able to get it to succesfully suspend execution under DingusPPC.

As a variant of attempt 3, I decided to make DingusPPC output the instruction stream that it was executing, hopefully there would be a clue where the problem was. To signpost the instructions that represented the implementation of StdTxMeas, I did two otherwise no-op divisions by specific literals (the MoveTo call is to ensure that the compiler didn’t decide to optimize them out).

GetPen(&pen);
pen.h = pen.h / 32749;

width = StdTxMeas(testString[0], testString + 1, &number, &denom, &fontInfo);

pen.v = pen.v / 32719;
MoveTo(pen.h, pen.v);

Checking the CodeWarrior dissembly I could see the expected li (load immediate) and divw (divide word) op codes:

// pen.h = pen.h / 32749;
    
000000A0: A861003A  lha      r3,58(SP)
000000A4: 38807FED  li       r4,32749
000000A8: 7C6323D6  divw     r3,r3,r4
000000AC: B061003A  sth      r3,58(SP)

I could thus look for the 32,749 divisor in DingusPPC’s divw implementation and then set a flag to trigger its built-in disassembler (and turn it off when the divison by 32,719 occurred). 

Out of this I had a list of 635 instructions that were executed, which was a lot, but still made for a starting point. Looking at counts of the types of instructions, there were 59 unique ones, with loads and stores being the most common. I figured that the most popular instructions were least likely to be buggy, otherwise more things would have been broken. Looking at the bottom of list, two stood out: nabs (negative absolute) and doz  (difference or zero). They both had no coverage in DingusPPC’s test suite³ and were obscure — they are actually part of the POWER instruction set that was the predecesor of PowerPC, and were only included in the PowerPC 601 chip to help with IBM’s migration to it.

Looking at the implementation of nabs, it seemed relatively straightforward — it would always set the sign bit to to 1, thus making the number always negative:

ppc_result_d = (0x80000000 | ppc_result_a);

However, negative numbers are not actually represented as positive numbers with the sign bit set, they use two’s complement. Using Julia Evans’s handy integer.exposed we can see the representation of 123, -123 and 123 with its sign bit flipped, and thus why this implementation is not right.

I switched it out to the more readable explicit sign test and negation:

ppc_result_d = ppc_result_a & 0x80000000 ? ppc_result_a : -ppc_result_a;

Things still weren’t working, but as I had previously mentioned, doz was also suspicious (and it was the instruction that was executed right before nabs). I noticed a copy/paste error in which register the result was being stored in, and once I fixed that, everything began to work!

Correctly rendered System 7.1.2 Finder, showing the contents of the "Power Macintosh CD" drive

This whole investigation took about a week’s worth of evening hacking time, and there was a bit of luck involved in the end. I very easily could have ended up on a wild goose chase looking at some of the other infrequent instructions, and given up before I got to nabs and doz. Or the implementation bugs could have been much more subtle. However, it worked out, and it was really satisfying to be able to contribute to the DingusPPC core.

  1. System enablers were used to patch in support for Macintosh models released after a base version of the OS shipped.
  2. Much of the Toolbox code was written in 68K assembly and so could not be easily ported when Apple made the 68K-to-PowerPC transition in 1994. Emulation was thus the solution not just for older software, but also for large parts of the OS too. It was not until the transition to Mac OS X in 2001 that the need for 68K emulation went away.
  3. The DingusPPC test suite is partially based on the one for the Dolphin emulator. Since that emulates a G3-class CPU, it does not need to worry about the quirks of first-generation PowerPC CPUs.