Upstream QEMU can run NetBSD/sparc64

Mark did a good job fixing OpenBIOS properties and cmd646 emulation in QEMU, so NetBSD started to get up to the user space. Then I sent my patch for emulation of short load instructions upstream, so with all this patches applied qemu-system-sparc64 can successfully boot NetBSD 6.1.4 (and likely all the other sparc64 versions):

       
Updating motd.
Starting sshd.
postfix: rebuilding /etc/mail/aliases (missing /etc/mail/aliases.db)
Starting inetd.
Starting cron.
Sat Aug  9 11 11:18:59 CEST 2014

NetBSD/sparc64 (netbsd614) (console)

login: root
Aug  9 11:19:27 netbsd614 login: ROOT LOGIN (root) on tty console
Last login: Fri Jan  1 13:21:40 2010 on console
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008, 2009, 2010, 2011, 2012
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 6.1.4 (GENERIC)

Welcome to NetBSD!

Terminal type is wsvt25.
We recommend that you create a non-root account and use su(1) for root access.
netbsd614#  ping 10.0.2.2
PING 10.0.2.2 (10.0.2.2): 48 data bytes
64 bytes from 10.0.2.2: icmp_seq=0 ttl=255 time=2.463 ms
64 bytes from 10.0.2.2: icmp_seq=1 ttl=255 time=1.994 ms

Have fun and please report the bugs to the qemu-devel@ mailing list.

Power management in SPARC & Solaris

Few days ago someone asked me in the comments to the How-To post, whether there is a way to reduce CPU consumption of the qemu-system-sparc. I assumed qemu lacks some device necessary for the power management. So, if we implement it, qemu-system-sparc won't take 100% of CPU all the time.

And then after looking at the qemu source I was really surprised: qemu does emulate the 'power-management' device. The reason why Solaris consumes so much CPU is Solaris itself! It's just doesn't have a driver for the 'power-management' device on SPARCStation-5. In all 'prtconf' outputs I could google out, there is a line:

power-management (driver not attached)

And then comes a bigger surprise: Linux perfectly supports CPU power management on SPARCStation-5. Even under qemu: after Debian/SPARC is booted, 'top' shows that qemu-system-sparc takes only 3% of a host  CPU.
 
How came that Sun didn't support CPU power management on their own sun4m machines?

P.S. By the way, NetBSD supports CPU power management too.

Of Course, It Runs NetBSD!™

NetBSD boot was almost a piece of cake. It tries to detect more things than Solaris and Linux, so I had to implement a couple of device registers more. At the first glance using more registers contradicts with the declared portability. On another hand, it would work on some modified/weird chipsets having non-standard interrupt controllers. Don't know if such chipsets were ever produced though.

NetBSD 4.0.1 (INSTALL) #0: Wed Oct  8 01:13:04 PDT 2008

builds@wb32:/home/builds/ab/netbsd-4-0-1-RELEASE/sparc64/200810080053Z-obj/home/builds/ab/netbsd-4-0-1-RELEASE/src/sys/arch/sparc64/compile/INSTALL

total memory = 256 MB

avail memory = 234 MB

timecounter: Timecounters tick every 10.000 msec

mainbus0 (root): QEMU,Ultra-3/2: hostid 80000000

cpu0 at mainbus0: SUNW,UltraSPARC @ 100.681 MHz, UPA id 0

cpu0: 32K instruction (32 b/l), 16K data (32 b/l), 512K external (64 b/l)

...

# ping 10.0.2.2

PING 10.0.2.2 (10.0.2.2): 56 data bytes

64 bytes from 10.0.2.2: icmp_seq=0 ttl=255 time=1.575 ms

64 bytes from 10.0.2.2: icmp_seq=1 ttl=255 time=1.150 ms

^C

----10.0.2.2 PING Statistics----

2 packets transmitted, 2 packets received, 0.0% packet loss

round-trip min/avg/max/stddev = 1.150/1.363/1.575/0.301 ms

#

Haven't found any CPU bugs so far, only interrupt processing in the serial port (not relevant to [Open]Solaris). Surprisingly sparc32 and sparc64 serial drivers diverge quite a lot.

Next stop - FreeBSD/sparc64.

Hiding the boot device from an OS

Suspecting a bug in SCSI subsystem, I was wondering how to prevent an OS from detecting the device it's booting from. At first it sounds weird, but it's possible for most of OS/distributions in the SPARC (and probably PPC) world, because they

• boot  init scripts from a RAM disk
• use IEEE 1275 device tree to find out available devices *

*) with exception of  Linux. At least the Linux uniform IDE driver tries to detect a hardware by probing.

For FreeBSD, NetBSD, OpenBSD and Solaris it's possible to hide the boot device from the OS install CD with the following commands:

ok cd scsi " unknown" name device-end
ok boot /unknown/sd@6,0:f

Use the Forth, Luke!

NetBSD vfs and MMU emulation

Since Solaris works perfectly, I turned again to NetBSD for the further test cases.

root on md0a dumps on md0b

root file system type: ffs

WARNING: clock gained 1005 days

WARNING: CHECK AND RESET THE DATE!

exec /sbin/init: error 8

init: trying /sbin/oinit

exec /sbin/oinit: error 2

init: trying /sbin/init.bak

exec /sbin/init.bak: error 2

init: not found panic: no init

cpu0: kdb breakpoint at 1362e60

Stopped in pid 1.1 (init) at 0x1362e64: nop

db>

Hmm what would be a reason for such a behavior? Sounds familiar, right? I hear you saying CPU math bug? Wrong. :)
The previous time I saw the message it was really a math bug. This time it's a MMU problem. Unlike SPARC v8, the v9 has a NFO mode for page mappings, which means No Fault Only. The 'only' part is crucial: all other page loads should fault.

Solaris knows what pages has been marked NFO (after all they can be marked by the OS only) and uses only the appropriate loads.

On the other side NetBSD vfs implementation explicitly provokes such faults to load RAM pages from a file. Had to read the NetBSD documentation to understand how its vfs works. The sources are not that good documented.

After fixing the bug (a sort of a counterpart to the one Tsuneo Saito mentioned on the mailing list), NetBSD gets a bit further:

Kernelized RAIDframe activated md0:

internal 5120 KB image area

root on md0a dumps on md0b

root file system type: ffs

WARNING: clock gained 1005 days

WARNING: CHECK AND RESET THE DATE!

erase ^?, werase ^W, kill ^U, intr ^C

The only part is missing here is the '#' prompt after the message. :) Working on it.

Going deeper into NetBSD problem

It looks like the documentation on the NCR89C100 chip (aka esp) is wrong or incomplete. Actually nothing new. Earlier I found that timer is definitely described wrong. But at least timer is described correctly in the Sun4m architecture manual, and for the scsi part the only source is the NCR document.

The subunits of the real machine are much more integrated than described. DVMA works differently compared to qemu, the expected interrupts are not triggered, and "Select with(out) attention" commands work differently too. At least in the mode NetBSD uses.

Starting to wonder why qemu is capable to boot anything at all. Looking at all this errors makes me feel like getting lost 100 meters away from the home. Or even in my own basement. Have to experiment further.

Bug in NetBSD 1.6 - 3.1 emulation

Was going to write that I couldn't imagine what did the NetBSD guys smoke during 5 years between the versions 1.6 and 3.1 inclusively. The NCR 53c9x SCSI chip (known as "esp" in SPARC and PPC machines) was not so uncommon back then. How could they introduce an instability and didn't notice it ?!?

The code from NetBSD 1.5.3:

NCRDMA_SETUP(sc, &sc->sc_cmdp, &sc->sc_cmdlen, 0, &dmasize);
//...
NCRCMD(sc, NCRCMD_SELATN | NCRCMD_DMA);
NCRDMA_GO(sc);

The code from NetBSD 1.6-3.1:

NCRCMD(sc, NCRCMD_SELATN | NCRCMD_DMA);
NCRDMA_SETUP(sc, &sc->sc_cmdp, &sc->sc_cmdlen, 0, &dmasize);
NCRDMA_GO(sc);

The code from NetBSD 4:

NCRDMA_SETUP(sc, &sc->sc_cmdp, &sc->sc_cmdlen, 0, &dmasize);
//...
NCRCMD(sc, NCRCMD_SELATN | NCRCMD_DMA);
NCRDMA_GO(sc);

See the difference? In the versions 1.6-3.1 the command is executed before the DMA is set up, so the SCSI controller may not get the command using DMA.

And then I googled for the expected bug reports and found none. Why could it work on the real hardware? Maybe some latency: if the latency of a SCSI controller was larger than a DMA controller, it might work? Maybe concurrency: if some other driver (for instance Ethernet) prepared DMA for itself, the SCSI driver could steal it? Maybe DMA didn't work for these versions at all and after the first attempt they switched to the PIO mode.

TME strikes back!

A great news for everyone who's interested in the 64-bit SPARC emulation (sun4u)!

After a nearly 3 years long pause Matt Fredette released the new version of TME - The Machine Emulator. Matt has skipped the sun4m emulation, which I think makes sense, because meanwhile qemu emulates sun4m pretty well (and fast!). Therefore the current list of the supported platforms is sun2, sun3, sun4c and sun4u. The only sun4u machine which tme can emulate is Ultra 1, so don't hold your breath for OpenSolaris support just yet.

But at least it can boot NetBSD 5. Also it uses the original OBP (not OpenBIOS) which implies that the emulation is pretty close to the hardware. It also seems to emulate cg6 graphic adapter which is much more powerful than qemu's tcx. Less powerful than Bob's cg14, but it's not yet included in the official git master anyway.

As for the OpenSolaris: it doesn't support the Ultra-1 and Ultra-2 machines. But! You can give a spin to Martux, the hacked OpenSolaris distribution which does have a support for the early Ultras.

Yet another bug in IRQ emulation

Trying to find out why NetBSD versions 1.6-3.1 do not boot, I found a bug in IRQ processing. After fixing it these versions still don't boot, :) but fail more gracefully.
Looks like there is another couple of bugs to go...

The World's fastest broken SS-5

Fixed a bug in the IRQ routing and now I have a machine gun, ho-ho-ho the World's fastest [broken] SparcStation-5! According to the Solaris 2.6 and Solaris7 output, it's faster than 1 GHz:

cpu0: FMI,MB86907 (mid 0 impl 0x0 ver 0x4 clock 1083 MHz)


Remember, last week I told that after fixing the performance problems I'm going to get back in the XXI century? Well, I lied. I did another quick stop in the past:

WARNING: clock gained 3987 days -- CHECK AND RESET THE DATE!

Guess, which OS is it?

A little improvement of SCSI disk emulation

NetBSD complained:

sd3: mode sense (4) returned nonsense; using fictitious geometry

SunOS 4.1.4 complains:

sd3: non-CCS device found at target 0 lun 0 on esp0

I was hoping that this is the same bug, so I tried to investigate. It's always easier to work when the sources are available. It turned out that NetBSD expects a block descriptor for mode pages, which wasn't implemented in qemu, and is probably mandatory for SCSI-2 disks. The specs are unclear: they don't explicitly say "optional" or "mandatory" in chapters 9.1.2 and 9.3.3.

Anyway, I implemented the block descriptor, and the NetBSD bug is gone. But this was in vain: the SunOS bug is still there, the Solaris bootblk problem is also un-affected.

Will send the patch upstream though. A small improvement is still an improvement.