Firefox on POWER9: the JIT of it

Four years ago, I reviewed a truly fully open source desktop computer, from operating system down to firmware: the Raptor Blackbird, built entirely around IBM’s POWER9 processor. The overall conclusion was that using was mostly an entirely boring experience, which was a very good thing – usually ideologically-fueled computers come with a ton of downsides and limitations for average users, but Raptor’s POWER9 machines bucked this trend by presenting a bog-standard, run-of-the-mill desktop Linux experience, almost indistinguishable from using an x86 machine.

Almost indistinguishable. The one thing that was missing from using desktop Linux on POWER9 was Firefox’ JIT, which meant that many websites, especially more complex ones, would bring the browsing experience down to a crawl. One area where this affected me quite a bit was our own WordPress backend, which is effectively unusable on Firefox without its JIT. The only other option was to use Chromium, which was fully ported to POWER9 – but I don’t like Chromium, and want to use Firefox to be able to share tabs, history, passwords, and so on.

Since then, back in 2021, things have improved. The ongoing effort to port Firefox’ JIT to POWER9, led by Cameron Kaiser, made a ton of progress, to the point where community Firefox builds with Kaiser’s JIT integrated became available through a dedicated Fedora copr. Sadly, the last build is from four months ago, and covers Firefox 128.14.0-1, an old ESR release. Since I recently set up the other machine Raptor sent to me – a Talos II workstation with two POWER9 processors – I was curious what the state of the POWER9 JIT effort was, so I inquired on the related bug report for Firefox.

Kaiser replied, and explained that due to a critical error with wasm against later versions of the JIT, as well as a change in his personal circumstances forcing him to work on this effort remotely – obviously not great for a client application like Firefox – there simply hasn’t been much progress, until last week (what a coincidence!).

Last week I took some time off work and dragged the JIT up to the current ESR. This compiles and links. However, although it passes the majority of the test suite, there are still too many serious failures to make it useable. I’m continuing work on this in whatever free time I actually have on my workstation. If I can restore test compliance in Baseline mode, this would suffice for a community third-party build like what Dan Horak generates now, since that is what is in 128. To get it in tree, however, I would also need to solve that critical wasm fault which manifested in the interim and fix the remaining gaps in the CodeGenerator to get it to a point of sufficient quality.

↫ Cameron Kaiser

There are two main problems at the moment that make it harder than it needs to be to work on this effort. First, the state of debugging tools on ppc64le – to which POWER9 belongs – is apparently not great, requiring Kaiser to step through thousands of instructions manually using gdb just to fix the last bug he discovered. That’s clearly deeply suboptimal, not fun, and not something somebody should spend their precious free time on.

At this point in the discussion, Raptor’s Timothy Pearson jumped in and noted that getting rr-debugger to work on POWER9 is something Raptor would be interested in, but it wouldn’t be cheap:

On the topic of the debugger (rr-debugger), while this isn’t on our internal roadmap at the moment it is something that Raptor could do under a development contract. The main question is whether there is enough interest to make that viable; the work is significant so the cost would probably be in the mid to upper 5 figures range (USD), assuming no major roadblocks are discovered.

When I was looking into it before I was fairly certain the PMU on POWER9 supports the overall structure of rr-debugger’s methods, and that our load-store idioms are generally compatible. The former is what stops it working on most arm64 devices IIRC, and the latter is relevant mainly to non-POWER RISC architectures.

↫ Timothy Pearson

Kaiser noted that while having rr-debugger available wouldn’t be a magic bullet, it would make the whole process a lot easier.

The second major issue is, of course, the same one as it always is for such niche efforts: a lack of manpower. According to Kaiser, there’s enough interest and awareness in getting Firefox’ JIT ported to POWER9, with the real problem being that there simply aren’t a lot of people with enough knowledge of both Firefox’ JIT and the modern ppc64le ISA. Understandably, Kaiser would like to avoid having to deal with people who are well-intentioned but don’t fully grasp the complexity of the undertaking at hand.

This is not exactly an easy effort, and it’s honestly downright amazing how far along the project already is. Even if it’s an older version, being able to run Firefox 128ESR on POWER9 with a working JIT makes such a huge difference to the overall desktop user experience, and I’m sure I speak for the entire POWER9 community when I say I’m incredibly grateful for it. Still, it would be amazing if we could find someone with just the right skillset to help Kaiser out, to be able to get the JIT stable enough again for community Firefox builds – and perhaps even look at what lies beyond: getting it upstreamed into Firefox as a whole.

The odds of finding that person are slim – if you’re into this sort of stuff, you’re most likely already aware of the POWER9 JIT effort – but who knows, maybe shining some renewed light on this task will make a difference. If you happen to have the right skillset and appreciate the complexities involved in this effort, you might want to reach out.