FreeBSD considering end of ppc64 support

FreeBSD is considering retiring powerpc64 prior to branching 16, which would make FreeBSD 15 the last stable version to support the architecture. (32-bit PowerPC is already dropped as of FreeBSD 14, though both OpenBSD and NetBSD generally serve this use case, and myself I have a Mac mini G4 running a custom NetBSD kernel with code from FreeBSD for automatic restart.) Although the message says "powerpc64 and powerpc64le" it later on only makes specific reference to the big-endian port, whereas both endiannesses appear on the FreeBSD platform page and on the download server.

I asked Warner Losh for clarification but he hasn't gotten back to me yet, so it's unclear if this is meant to apply to just the BE port or both. Unlike the overall industry mood generally, especially from you underannuated whippersnappers who deal with the doublethink of big-endian math in your head but little-endian code in your repos, the BSDs have historically been friendly to big-endian hardware as a means of ensuring code correctness. Indeed, his note acknowledges this. One sticking point seems to be Power ISA's own unique architectural features but I suspect it is due at least as much to the higher cost and poorer availability of the hardware, which has admittedly been thin on the ground lately.

My preference in server operating systems is a BSD-variant and FreeBSD was going to be my own migration plan when I retire my POWER6 running AIX (big-endian, natch), but this news has made me consider OpenBSD/powerpc64 instead. This port is a little suboptimal in that it states it does not presently run under a hypervisor, but it is currently well-supported and specifically tested on Raptor hardware, which is going to be the majority of the ecosystem. Of course, OpenBSD is also a famously opinionated operating system, so there may be a little bit more of a learning curve for me personally, whereas I already have experience with FreeBSD on shared x86_64 hosting and my other BSD systems all run NetBSD (on /mac68k, /cobalt, /macppc, /alpha, /hpcmips and /hpcsh). Another possibility is to try to maintain security patches and fixes on the remnant of FreeBSD 15, though I have such limited time these days I'd prefer not to pour it into another endeavour with an obvious future dead end. I'd dearly love a NetBSD option, but there isn't a NetBSD port that runs on POWER9.

I do think it's a little unfair to cite the user base as being less than "x86 or armv7" when these were extremely common architectures in their day, and a lot of people are probably still running FreeBSD on older examples to get the most money they can out of their hardware. I can't imagine their RISC-V numbers are very high either but no one is saying that should go away, and Power ISA is still very relevant in today's big-iron shops. If and until "low-end" Power11 desktop hardware is available, we'll want all the operating system support we can get too.

OpenBSD 7.5

OpenBSD 7.5 is out with multiple kernel and SMP improvements (we love SMP improvements on our multicore beasts), more hardware support, and LibreSSL 3.9.0, OpenSSH 9.6/9.7, and LLVM-clang 16.0.6. The only headliner Power ISA specific improvement to the big-endian powerpc64 port is a smoother upgrade process, but all the other advancements are welcome too. Download from any of the many mirror sites.

OpenBSD 7.4

OpenBSD 7.4 is released, an increasingly mature BSD option for OpenPOWER systems. While there are relatively few improvements this time around specifically for the powerpc64 port, this release does have improvements for SMP (a big deal for our pervasively SMT cores), performance and security upgrades with the virtual memory manager, and updates and bugfixes to userland. You can download it, or read the entire changelog.

FreeBSD 13.2

And hot on the heels of the latest OpenBSD release is the latest FreeBSD iteration, 13.2-RELEASE. FreeBSD has a longer track record on OpenPOWER and in my cursory estimates is the most commonly installed BSD on modern Power ISA. One big jump is that the bhyve hypervisor now supports more than 16 virtual CPUs and by default can create the same number of vCPUs as physical CPUs, which is quite useful to us once you get away from the smallest single-4 machines given all our cores are SMT-4. Additionally, for those of you running FreeBSD on a VM (such as an LPAR or under KVM), nested POWER9 radix MMU mappings are now supported on the pseries flavour, substantially reducing hypercall overhead. The Linux compatibility ABI has also been expanded and on the security side ASLR is now enabled for all 64-bit executables by default, configurable through proccontrol. Downloads are available for big-endian and little-endian. Note that the release notes indicate that all PowerPC and Power ISA releases right now must run kldxref /boot/kernel manually after an upgraded successful kernel and world installation.

OpenBSD 7.3

OpenBSD 7.3 is released. While most of the improvements are not specific to Power ISA, there's a lot we benefit from, including many kernel calls which are now "lock-free" (improving SMP performance) like mmap(2) and select(2), more device support, immutable permissions on address ranges to prevent permissions from being changed in the future — much of a running program's static address space like stack, code and most libraries is now automatically immutable — and support for execute-only memory on both Power ISA and the PowerPC 970 ("G5"). LibreSSL is updated to 3.7.2, OpenSSH is updated to 9.3, and the OS ships with LLVM/clang 13.0.0 and Perl 5.36.0. Download and install when ready, Puffy.

OpenBSD 7.2

The latest release of OpenBSD is available, the new 7.2. Although there are few, if any, Power-specific changes, there are many welcome general ones including multiple improvements in SMP and updated graphics and hardware drivers. LibreSSL and OpenSSH are also updated. Note that the powerpc64 port remains big-endian only — which I have to admit is lovely — and the changelog page only gives "XXXX" for number of prebuilt packages available for 32-bit powerpc and 64-bit powerpc64. Here is a complete list of changes and download mirrors.

FreeBSD 13.1

FreeBSD 13.1 is now available, notable as it comes in both big-endian and little-endian flavours depending on how you swing (as well as 32-bit ports for Power Macs and the AmigaOne A1222). The biggest change in this version is that 13's shift to the PowerPC ELF ABIv2 makes it binary-incompatible with 12.x (so you may need to rebuild or relink; the release notes also suggest doing kldxref /boot/kernel after successful installation or upgrade). However, there are also fixes for the bootloader on the LE port, OpenSSL performance improvements, a serial console fix for the BE port, a fix for running FreeBSD with HPT superpages enabled on QEMU with TCG (if you're trying before you're buying), and — particularly of relevance to those of us on OpenPOWER hardware — a fix for the AST2500 console on bootup with recent OpenBMC firmware. It's a big update but one that makes OpenPOWER an even better citizen on FreeBSD (now if only NetBSD would get with the program, because at this point it's just embarrassing). Read the release notes, or download.

OpenBSD 7.0

OpenBSD 7.0 is available, compatible with Raptor workstations in big-endian mode as well as "expected to be" with IBM PowerNV hardware generally. New powerpc64-specific improvements include MSI-X support, a fix for page faults under recursive locking, a bump in the maximum data size to 32GB, and support for the dynamic tracer. This is on top of better GPU support, additional driver and device support, updates to OpenSMTPD, LibreSSL and OpenSSH, and lots of new port packages. You can boot OpenBSD directly from Petitboot and install over the network; download mirrors are worldwide.

OpenBSD 6.9

Great past few days for new OpenPOWER operating system support: now the newest release of OpenBSD is available as well to complement Ubuntu and Fedora updates earlier this week. Many improvements have landed to the big-endian powerpc64 port since its début in OpenBSD 6.8, most notably framebuffer support for the ASPEED BMC, workarounds for addressing AMD GPUs over PCIe, power-saving mode for POWER9, and IPMI on PowerNV systems (which is all Raptor-family machines and quite a few others). General to all ports include an additional privacy guard for video devices (similar to the existing one for audio) among other webcam fixes, encrypted RAID-1, performance improvements to SMP, lots of additional hardware drivers, security fixes and updates to networking and crypto.

With this release, your BSD choices on OpenPOWER just got more solid between this and the mature FreeBSD port. Again, the real shame is why there's still no support for OpenPOWER in NetBSD.

FreeBSD 13 and Guix for OpenPOWER

After a bit of downtime, we're back. And cool stuff has happened in our absence, the most notable being additional improvements to the increasingly mature OpenPOWER port of FreeBSD. 13-RELEASE, among other changes, officially introduces the 64-bit little-endian port (previously exclusively big-endian, which is still supported), experimental radix MMU support for POWER9 (hashed page tables are of course supported everywhere), XIVE interrupt support on POWER9 (about 10% faster), optimized memcpy(), memmove() and like-minded standard functions, and many stability and performance improvements. The releases notes say that "performance during bulk -a package building is at least 60% higher" which is very impressive. ISOs are available from their download server.

In addition, ppc64le support has been merged to the GNU Guix source tree, meaning with the next expected version 1.2.1 you'll hopefully be able to get a pre-built copy. It's been in development for several months and now it appears to be finally approaching reality. Like Guix the package manager, the GNU Guix System's most notable feature is its declarative service and package configuration, all on top of the GNU Shepherd init system and (right now) Linux 5.9. Currently there is still a reproducibility issue with gcc, rust is still at least somewhat experimental (which is relevant for librsvg) and many packages have not been tested. Still, since the Talos II and T2 Lite are GNU Respects Your Freedom systems, now you can run another GNU-free OS on them too and sooner than you think.

Updates: Fedora 33, FreeBSD 12.2, Ubuntu 20.10

Hot on the heels of Ubuntu 20.10 and 20.04.1 LTS (download the server flavour, and convert it to desktop if you like) comes Fedora 33. Ubuntu 20.10 upgrades to kernel 5.8, GNOME 3.38, QEMU 5 and OpenStack Victoria with an installer fix for OpenPOWER; Fedora 33 remains on 5.8 (5.9 likely to follow) but also includes GNOME 3.38, glibc 2.32 and LLVM 11, and also defaults to btrfs on Workstation (watch out if you change to a 4K page size; Fedora uses 64K pages and filesystems generated on one are not currently compatible with the other). As previously mentioned Fedora is important to me personally because it's what I run on my own T2 and Blackbird, so once the packages and late breaking changes settle down I will do a mini-review (as I did for F32), but the change I've been waiting for (128-bit long doubles) is still not in F33 as they wait on glibc changes (maybe glibc 2.33).

And if you like your OpenPOWER systems but don't like Linux, FreeBSD 12.2 is out as well, with multiple security, bugfix and functionality upgrades for a wide variety of PowerPC and OpenPOWER-based systems. Big-endian is well-tested and little-endian is coming along (and snapshots should finally be in -CURRENT by the time you read this).

OpenBSD officially available for ppc64

OpenBSD 6.8 is now available and with it the first official release of the big-endian ppc64 port (which they call powerpc64). The port is specifically advertised for PowerNV machines (i.e., bare metal) with POWER9, which naturally includes the Raptor family but should also support IBM PowerNV systems as well. POWER8 support is described as "included but untested."

The installation directions are still not fully complete, though Petitboot should be able to start the installer from pretty much any standard medium, and the installer experience should be stock from there. What's more, it looks like a good selection of pre-built packages is available, though some large applications are still missing like Firefox (WebKit is apparently available). The missing packages seems to be similar to what is missing for their 32-bit powerpc flavour, so this is not unexpected.

With OpenBSD's release and FreeBSD's well-regarded history, this leaves only NetBSD — ironically the BSD with the most emphasis on portability, and my personal preference — as the last major cross-platform BSD yet to arrive on OpenPOWER. Given OpenBSD and NetBSD's genetic history, however, this release makes future NetBSD support for OpenPOWER much more likely.

FreeBSD swings both ways

They say there's an xkcd for everything, but me, I say it's Friends GIFs. Anyway, hat tip to developer Piotr Kubaj who reports that, if you don't like big endian and cannot lie, FreeBSD's covered you got with a new little endian ppc64le port to complement the existing (and by now practically mature) big endian ppc64 flavour.

Raptor themselves actually give material support to the project by providing a remote instance for development, powering a build server that continuously runs poudriere bulk -a to test ports. Plus, looking in the source tree, the commits to add little-endian support are all tagged as "Sponsored by: Tag1 Consulting, Inc." This company apparently has OpenPOWER alumni from the Oregon State University Open Source Lab (.pdf). It's nice to see the cross-pollination at work!

Although there are no .iso images yet, they should start appearing with the -CURRENT snapshots next week. Note that official ports support doesn't exist yet either, so you'll need to compile packages on your own for the moment, and there are other minor to moderate deficiencies relative to the big-endian port which are still being rectified. Still, choice is a good thing, especially since per Piotr there are no plans to decommission the big-endian port and both will coexist. How's that for playing on both teams?

When will OpenPOWER OpenBSD be now? Now.

We were delighted by the tease that OpenBSD is moving to OpenPOWER (although it is officially big-endian powerpc64, it requires OPAL, so a POWER8 is minimally required). Well, now you can try it out: a powerpc64 snapshot is now available with most of the standard binary distribution sets. The installation documentation is pretty much copy-pasta — I doubt very much that 64-bit PowerPC is supported on AMD Opteron, and I would be impressed to learn that the Pinebook Pro is OpenPOWER — but you should be able to boot from the miniroot (flash it to a USB drive using dd bs=1m) and manually setup and copy the file sets over. Curiously, the X11 distribution sets do not appear to be built yet, so you may be restricted to a text boot and/or the serial console. When I get my spare Talos II back up and running I intend to give this a full shakedown, since this would be a great basis for finally having NetBSD on OpenPOWER (my personal BSD of choice). The fact it doesn't right now is a great shame to an OS that is supposed to run everywhere but doesn't on one of the most open platforms anywhere.

FreeBSD on POWER

We haven't covered BSD a great deal in this blog even though I personally run NetBSD on three systems myself (two of which are in regular service), mostly because my system and I suspect the majority of the OpenPOWER install base is on Linux. However, FreeBSD 11.3 is now officially released and has fairly good support for 32-bit and 64-bit PowerPC on Power Mac hardware, so it's worth pointing out that 12.0 (and 13.0) has also been tested on the Blackbird and thus should also work on the Talos II. However, on the PowerPC wiki page -CURRENT is recommended for Blackbird, 12.0 is mandatory for OpenPOWER (thus 11.x won't work and presumably won't ever work), and X11 is currently listed "on Power8/Power9 [as] still a work in progress." Nevertheless, POWER8 systems also work, hardware support is improving and the OS offers another big-endian option for people preferring to run their systems that way, so hopefully Justin or Mark who are more versed in the FreeBSD world than I am have some comments about how well it works for others to explore.