Summary

Two defects in the MPU-401 MIDI input path. One can hang the emulator, the other sends the wrong bytes to the MIDI output port and can crash 86Box when no output device is open.

MPU401_RecQueueBuffer (src/sound/snd_mpu401.c:196) never returns once the record queue is full: cnt is only incremented inside if (mpu->rec_queue_used < MPU401_INPUT_QUEUE), so when that test fails cnt < len can never become false. Only the guest reading the data port (:769) and the block at the end of this same function drain the queue, and the spin reaches neither. MPU401_ClrQueue (:235) and the SysEx abort path (:1391) do zero rec_queue_used, but two call sites are reachable from the emulation thread (:482 and :490, in the command write handler), where the spinning thread is the one that would have to issue that reset.

Filling the queue does not need an unusual guest. In intelligent mode any real-time byte above 0xF8 is queued as two bytes with no room check (:1555-1560), and RtMidi is deliberately configured not to ignore active sensing (src/sound/midi_rtmidi.cpp:188). With the guest not reading the port, a controller sending active sensing every 300 ms fills the 1024-byte queue on message 512, about two and a half minutes, and the call that queues the next message never returns.

The loop now stops when there is no room and drops the rest of the event, which is what MPU401_InputSysex already does when the queue is full (:1401-1409), and dropping is the only alternative to blocking here. InputSysex itself is unchanged, since it already clamps to the free space. This does not conflict with the MIDI input queuing work mentioned in #6266: the loop is unbounded whatever the threading, and a rework of the input queue would replace it along with the queue.

The second fix is in midi_raw_out_rt_byte (src/sound/midi.c:310). It checks midi_in, then calls play_msg, which dereferences midi_out with no check (midi.c:219). midi_out is NULL whenever MIDI Out is None or the host port failed to open, and the settings dialog allows an MPU-401 with MIDI In and no MIDI Out. The configuration attached to #6266 is exactly that, midi_in_device = midi_in under [Sound] and no midi_device key.

The formatting is wrong too, whenever the device's Real time option is on. val << 24 puts the status byte at offset 3 of the int on a little-endian host, so rtmidi_play_msg reads msg[0] as zero, takes midi_lengths[0] which is 3, and sends three zero bytes instead of the single F8, FA, FB or FC (src/sound/midi_rtmidi.cpp:51 and :59-64). Anything slaved to the emulated MPU-401 clock never starts, stops or syncs. midi_raw_out_byte (midi.c:334) already gets both parts right, so this now guards midi_out and uses midi_out->midi_rt_buf the same way, leaving the MIDI In policy checks above it alone. That removes the only write to midi_t.midi_cmd_r (src/include/86box/midi.h:61), which was already write-only. I left the field in place to keep the diff to the bug, happy to drop it if you would rather.

Checked with two standalone programs rather than a full build, since Qt is not available here. The first runs the fill loop verbatim with the real MPU401_INPUT_QUEUE and the real call shape from the active sensing path, nothing draining the queue, with an iteration cap so it reports instead of hanging: current code reaches rec_queue_used=1024 on message 512 and does not terminate on 513, the patched loop returns from every call. The second prints what play_msg receives, 00 00 00 f8 at length 3 before and f8 00 00 00 at length 1 after. Both files compile with clang -fsyntax-only -Wall -Wextra with exactly the same warnings as before, and the changed lines are clang-format clean.

Neither fix closes #6266. The IRQ 2 crash there is the ACPI SCI redirect already identified in the thread, and that report has realtime = 0. The overflow spin is a plausible cause of the "MIDI input simply ceases" behaviour described in the same report, but I have not reproduced that here.

Checklist

• I have tested my changes locally and validated that the functionality works as intended

References

• System Real-Time messages are single bytes, 0xF8 to 0xFF: https://midi.org/summary-of-midi-1-0-messages
• Related report, not closed by this pull request: MIDI Input Causes Crash #6266