From SCSI to PCI

20 years ago SCSI devices ruled the world. Reading the NCR53c810 manual, I see that it was basically a computer in a computer. It can be programmed to transfer data from/to/between the disks without using the CPU at all.

Trying to understand what happens in the NCR/LSI script:

lsi_scsi: Select LUN 0
lsi_scsi: Extended message 0x1 (len 3)
lsi_scsi: SDTR (ignored)
lsi_scsi: SCRIPTS dsp=81c3b924 opcode 80080000 arg 81c3b9a4
lsi_scsi: Jump to 0x81c3b9a4
lsi_scsi: SCRIPTS dsp=81c3b9a4 opcode 870b0000 arg 81c3b9c4
lsi_scsi: Compare phase 2 == 7
lsi_scsi: Control condition failed
lsi_scsi: SCRIPTS dsp=81c3b9ac opcode 860a0000 arg 81c3b94c
lsi_scsi: Compare phase 2 == 6
lsi_scsi: Control condition failed
lsi_scsi: SCRIPTS dsp=81c3b9b4 opcode 98080000 arg 00000022
lsi_scsi: Interrupt 0x00000022
...
lsi_scsi: SCRIPTS dsp=81c3bb6c opcode 0e000002 arg 81c3bdb4
lsi_scsi: MSG out len=2
lsi_scsi: Select LUN 0
lsi_scsi: MSG: ABORT tag=0x0
lsi_scsi: SCRIPTS dsp=81c3bb74 opcode 80080000 arg 81c3bbbc
lsi_scsi: Jump to 0x81c3bbbc
lsi_scsi: SCRIPTS dsp=81c3bbbc opcode 60000008 arg 00000000
lsi_scsi: Clear ATN

Looks like it aborts if the selected SCSI target doesn't change phase to MSG_OUT or MSG_IN. So I implemented a hack for SDTR reply and it doesn't abort here. But indeed it's a red herring. AIX can also work with the devices which do not support the synchronous or wide transfers.

The actual problem happens later:

lsi_scsi: SCRIPTS dsp=81c43444 opcode c0000004 arg 010000dc
lsi_scsi: memcpy dest 0x81c435fc src 0x010000dc count 4

Or, with a bit more enhanced logging:

lsi_scsi: memcpy dest 0x81c475fc (Mem) src 0x010000dc (IO) count 4
lsi_mem_read, address_space_read status 2
lsi_scsi: the first 4 bytes: 00 00 00 00

It tries to read the port 0x10000dc and save it. QEMU doesn't have anything at the port 0x10000dc, so no wonder the NCR script fails. But what is supposed to be there? The Motorola Ultra 603/Ultra 603e/Ultra 604 Programmer’s Reference Guide suggests it must be the PCI I/O space.

So looks like I've learned enough of NCR/LSI script. Time to see how the PCI bus mastering is supposed to work on this machine.

More fun with AIX cfgncr_scsi

<in the previous part>... doesn't find SCSI disks. Here it is tricky, it may be a problem with the interrupt routing, or DMA or SCSI host emulation...

... or a bug in the AIX driver itself.

As AIX 4.2 tries to perform scsi inquiry, that's what happens in the QEMU log:

(qemu) lsi_scsi: Write reg ??? ac = e4
lsi_scsi: Write reg ??? ad = 38
lsi_scsi: Write reg ??? ae = c4
lsi_scsi: Write reg ??? af = 81

The register at 0xac-0xaf is DSA Relative Selector (DRS). Is known to qemu, but seems to be not used in any operations.

The newer LSI53c1010-66 manual says:

"This register supplies AD[63:32] during Table Indirect
Fetches and Load/Store Data Structure Address (DSA)
relative operations"

So, maybe just add the support of this register to QEMU and allow the 64 bit DMA transfers, right?

Wrong. The write to this register is the last write and it doesn't start any SCSI command. Let's look where it happens:

p8xx_start_chip:
...
   0x018ff854:  stw     r8,-4(r7)
   0x018ff858:  li      r4,44         ; 0x2c
   0x018ff85c:  b       0x18fe348     ; p8xx_write_reg <= write happens here

The register r4 is 0x2c, but the procedure writes to 0xac. Weird.

Let's look at the other registers:

0x018fe368 in ?? ()
(gdb) info registers r3 r5 r4
r3             0x18f5000        26169344
r5             0x31000080       822083712
r4             0x2c     44

What's that 80 at the end of r5? 0x80 + 0x2c is 0xac. Coincidence? Don't think so.

So, what happens here is the driver tries to write 0x2c, but the bus is shifted, so it hits 0xac. After some chasing I found where this shift is coming from:

p8xx_config:
...
   0x018fdb9c:  bl      0x1909938
   0x018fdba0:  lwz     r2,20(r1)
   0x018fdba4:  cmpwi   cr1,r3,19
   0x018fdba8:  beq     cr1,0x18fdbc8
   0x018fdbac:  li      r8,1
   0x018fdbb0:  stb     r8,256(r28)
   0x018fdbb4:  li      r3,0
   0x018fdbb8:  bl      0x1909938
   0x018fdbbc:  lwz     r2,20(r1)
   0x018fdbc0:  lwz     r8,160(r28)
   0x018fdbc4:  b       0x18fdbd0
   0x018fdbc8:  stb     r29,256(r28)
   0x018fdbcc:  lwz     r8,160(r28)
   0x018fdbd0:  lis     r11,4096
   0x018fdbd4:  addic   r10,r8,128    ; this is the 0x80 I'm looking for
   0x018fdbd8:  li      r8,-1
   0x018fdbdc:  rlwinm  r31,r26,1,15,30
   0x018fdbe0:  addic   r23,r28,10580
   0x018fdbe4:  stw     r10,252(r28)
   0x018fdbe8:  li      r25,1
   0x018fdbec:  addi    r30,r28,0
   0x018fdbf0:  stwu    r25,10512(r30)
   0x018fdbf4:  stw     r8,10588(r28)
   0x018fdbf8:  lwz     r8,10500(r28)
   0x018fdbfc:  stw     r11,10520(r28)
   0x018fdc00:  stw     r10,10532(r28) ; and here it is stored
...

It is added and stored unconditionally. If I drop this addic, something different happens:

(qemu) lsi_scsi: Write reg DSP0 2c = e4
lsi_scsi: Write reg DSP1 2d = 58
lsi_scsi: Write reg DSP2 2e = c4
lsi_scsi: Write reg DSP3 2f = 81
lsi_scsi: SCRIPTS dsp=81c458e4 opcode 41000000 arg 81c45a44
lsi_scsi: Selected target 0
lsi_scsi: SCRIPTS dsp=81c458ec opcode 78370000 arg 00000000
lsi_scsi: Read-Modify-Write reg 0x37 MOV data8=0x00 sfbr=0x00
...

Why would it work on the physical hardware? I guess because the addresses are aliased. Pretty similar to the le bug in Solaris.

So, it's not that QEMU has some unimplemented registers. In this case it has too many implemented ones.

On the other hand, it still doesn't detect the scsi disk, so maybe it has not just too much features, but too few as well...

/Stay tuned

Milestone: my OFW boots AIX on Powerstack II Utah

Created residual data for the PCI bus, serial port and SCSI adapter. This is the minimal set to boot AIX. And AIX is booting using this residual data, not the hard coded ones. So now I have a reference firmware which works on a physical machine. Here is the complete boot log (mostly for search engines, and digital archaeologists):

MOT PowerStack2 (e0), Serial #0, 62 MiB memory installed
Open Firmware , Built  August 25, 2017 23:09:26
Copyright (c) 1995-2000, FirmWorks.
Copyright (c) 2014,2017, Artyom Tarasenko.

ok boot /scsi/disk@6 -s prompt

Boot device: /scsi/disk@6  Arguments: -s prompt
 0) - DISABLE_PARITY                   16) - ENABLE_END_STOP
 1) - DISABLE_DCACHE                   17) - ENABLE_DEBUG
 2) - DISABLE_ICACHE                   18) + DISABLE_VME
 3) - DISABLE_L2                       19) + ENABLE_BH_IDE_DMA
 4) - DISABLE_SSCALAR                  20) + WINBOND_PATCH
 5) - DISABLE_BHIST                    21) - MANUAL_SCSI_TYPE
 6) - DISABLE_CPU_EMCP                 
 7) - DISABLE_EAGLE_CF_DPARK           
 8) - DISABLE_EAGLE_CF_APARK           
 9) - DISABLE_LEDS                     
10) - EAGLE_ERR_STATUS_RESET           
11) + DISABLE_MASTER_ABORT             
12) - AIX_USES_BUG                     
13) + JUNO_DISCONTIGUOUS               
14) - LLDB_STOP                        
15) - SERVICE_MODE                     31) - DISABLE_HARDSTOPS
Enter bit # to toggle (just <CR> to end): 17
 0) - Top level debug - function names 
 1) - Main line debug messages         
 2) - Subroutine internal messages     
 3) - PCI Bridge settings              
                                       
                                       
                                       
10) - GEV Data debug                   
                                       
12) - IPLCB data                       
                                       
                                       
15) - IPL control block offsets        
Enter bit #(s) to toggle, '*' for ALL enabled, 'C' to clear ALL, 
or just <CR> to return): *
 0) - DISABLE_PARITY                   16) - ENABLE_END_STOP
 1) - DISABLE_DCACHE                   17) + ENABLE_DEBUG
 2) - DISABLE_ICACHE                   18) + DISABLE_VME
 3) - DISABLE_L2                       19) + ENABLE_BH_IDE_DMA
 4) - DISABLE_SSCALAR                  20) + WINBOND_PATCH
 5) - DISABLE_BHIST                    21) - MANUAL_SCSI_TYPE
 6) - DISABLE_CPU_EMCP                 
 7) - DISABLE_EAGLE_CF_DPARK           
 8) - DISABLE_EAGLE_CF_APARK           
 9) - DISABLE_LEDS                     
10) - EAGLE_ERR_STATUS_RESET           
11) + DISABLE_MASTER_ABORT             
12) - AIX_USES_BUG                     
13) + JUNO_DISCONTIGUOUS               
14) - LLDB_STOP                        
15) - SERVICE_MODE                     31) - DISABLE_HARDSTOPS
Enter bit # to toggle (just <CR> to end): 
0x4fd0 Hints relocation
0x5000 SoftROS start() after relocation
0x39728 SoftROS end after relocations
0x3951c SoftROS start of bss
0x39727 SoftROS end of bss
0x39750 Current sbrk(0)
0x596ac Current stack
Space reserved for kernel when IPLCB is being built @ 0x59728
Original bootimage located @ 0x400000
hi->signature = 0x4149584d
hi->resid_data_address = 0x3dd7970
hi->bss_offset = 0x3951c
hi->bss_length = 0x20c
hi->jump_offset = 0x38c
hi->load_exec_address = 0x400430
hi->header_size = 0x400
hi->header_block_size = 0x25e
hi->image_length = 0x4b75b
hi->Spare = 0x3cb419c
hi->res_mem_size = 0x0
hi->mode_control = 0xdead00c0
LED(MOTLED_CHECKING_HARDSTOPS)=0x130c
Magic is 0x01DF0004 
Image size .............. 0x0035A000
Boot image loaded at .... 0x0044BC00
Saved address for jump .. 0x0000038C
LED(MOTLED_INVALID_BOOT_IMAGE)=0x1310
LED(MOTLED_FIRST_KERNEL_MOVE)=0x1308
LED(MOTLED_HARDWARE_INIT)=0x130b
mot_gencmd.c ====> pj_motorola
mot_gencmd.c ====> Machine Check Pin was disabled by F/W

LED(MOTLED_ENABLING_CPU_EMCP)=0x1318
LED(MOTLED_ENABLING_DCACHE)=0x131a
LED(MOTLED_ENABLING_604_SSCALAR)=0x131c
LED(MOTLED_ENABLING_604_BHIST)=0x131d
LED(MOTLED_HARDWARE_INIT_COMPLETE)=0x1320
LED(MOTLED_IPLCB_INIT)=0x1309
iplcb_init.c ====> - iplcb_init()
iplcb_init.c ====> - mem_find()
Mem_addr = 0x3c74000, byte_index = 0x1ef, bit_index = 0x4
Returned from mem_find:
IPLCB addr: ..... 0x03C74000 len = 49152
DMA buffer addr:  0x00FF8000
Memory bitmap addr0x03C7FE10
Serial # from residual data:  4d 4f 54 30 45 32 33 34 41 43 20 20 20 20 20 20
nvram_addr = 0x0074, nvram_data = 0x0077
Name = board-init?  <--> Value = true  Match = FALSE
Name = use-default-vals?  <--> Value = true  Match = FALSE
Name = edo-memory?  <--> Value = false  Match = FALSE
Name = pboot-probe?  <--> Value = false  Match = FALSE
Name = pboot-device-default  <--> Value = fdisk0 hdisk0 enet0  Match = FALSE
Name = fcode-debug?  <--> Value = true<FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF>
<FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF>
<FF><FF>  Match = FALSE
Name = fw-boot-device  <--> Value = /pci@80000000/pci1000,3@2,0/harddisk@6,0  Match = TRUE
The IPLCB previpl_device string is:
        !!^A/pci@80000000/pci1000,3@2,0/harddisk@6,0
processor = 0x00000009
presoftros.c ====> - __mot_eth_addr()
Memory address of IPL Control Block = 0x03C74000
Directory: ......... 0x03C74080    offset: 0x00000080
IPL_info: .......... 0x03C742E0    offset: 0x000002E0
System area: ....... 0x03C74878    offset: 0x00000878
Buc Data area: ..... 0x03C749F4    offset: 0x00000914
Processor data area: 0x03C74A64    offset: 0x00000A64
Network data area: . 0x03C74D7C    offset: 0x00000D7C
Memory data area: .. 0x03C7677C    offset: 0x0000277C
L2_cache data area:  0x03C768B4    offset: 0x000028B4
Residual data area:  0x03C76974    offset: 0x00002974
ros_table area: .... 0x03C7D3E0
NVRAM cache area: .. 0x03C7D4C8    offset: 0x000094C8
Parameters passed to kernel boot image:
0x3c74000, 0xbe10, 0x4000, 0x1f0
LED(MOTLED_IPLCB_DUMP)=0x130a
IPLD.ipl_info_offset = 0x2e0
IPLD.ipl_info_size = 0x598
IPLD.system_info_offset = 0x878
IPLD.system_info_size = 0x9c
IPLD.buc_info_offset = 0x914
IPLD.buc_info_size = 0x150
IPLD.processor_info_offset = 0xa64
IPLD.processor_info_size = 0x318
IPLD.mem_data_offset = 0x277c
IPLD.mem_data_size = 0x138
IPLD.l2_data_offset = 0x28b4
IPLD.l2_data_size = 0xc0
IPLD.bit_map_offset = 0xbe10
IPLD.bit_map_size = 0x1f0
(IPLD.processor_info_size == sizeof(PROCESSOR_DATA)) failed
IPLD.user_struct_offset = 0x9380
IPLD.user_struct_size = 0x10
user_info->user_data_offset = 0x9390
user_info->user_data_len = 0x50
IPLD.nvram_cache_offset = 0x94c8
IPLD.nvram_cache_size = 0x2000
ipl_info->model = 0x80000e0
ipl_info->ram_size = 0x3e00000
ipl_info->bit_map_bytes_per_bit = 0x4000
ipl_info->ros_entry_table_ptr = 0x3c7d3e0
ipl_info->ros_entry_table_size = 0xe8
ipl_info->nvram_section_1_valid = 0x1
ipl_info->vpd_processor_serial_number = "00E20000"
ipl_info->previpl_device[0] = 0x21
ipl_info->previpl_device[1] = 0x21
ipl_info->previpl_device[2] = 0x1
ipl_info->previpl_device[3] = 0x2f
ipl_info->previpl_device[4] = 0x70
ipl_info->previpl_device[5] = 0x63
ipl_info->previpl_device[6] = 0x69
ipl_info->previpl_device[7] = 0x40
ipl_info->previpl_device[8] = 0x38
ipl_info->previpl_device[9] = 0x30
ipl_info->previpl_device[10] = 0x30
ipl_info->previpl_device[11] = 0x30
ipl_info->previpl_device[12] = 0x30
ipl_info->previpl_device[13] = 0x30
ipl_info->previpl_device[14] = 0x30
ipl_info->previpl_device[15] = 0x30
ipl_info->previpl_device[16] = 0x2f
ipl_info->previpl_device[17] = 0x70
ipl_info->previpl_device[18] = 0x63
ipl_info->previpl_device[19] = 0x69
ipl_info->previpl_device[20] = 0x31
ipl_info->previpl_device[21] = 0x30
ipl_info->previpl_device[22] = 0x30
ipl_info->previpl_device[23] = 0x30
ipl_info->previpl_device[24] = 0x2c
ipl_info->previpl_device[25] = 0x33
ipl_info->Power_Status_and_keylock_reg = 0x3
buc_info_ptr->num_of_structs = 0x3
buc_info_ptr->index = 0x1
buc_info_ptr->struct_size = 0x70
buc_info_ptr->bsrr_offset = 0x0
buc_info_ptr->bsrr_mask = 0x0
buc_info_ptr->bscr_value = 0x0
buc_info_ptr->cfg_status = 0x2
buc_info_ptr->device_type = 0x5
buc_info_ptr->num_of_buids = 0x0
buc_info_ptr->buid_data[0].buid_value = 0xffffffff
buc_info_ptr->buid_data[0].buid_Sptr = 0x0
buc_info_ptr->buid_data[1].buid_value = 0xffffffff
buc_info_ptr->buid_data[1].buid_Sptr = 0x0
buc_info_ptr->buid_data[2].buid_value = 0xffffffff
buc_info_ptr->buid_data[2].buid_Sptr = 0x0
buc_info_ptr->buid_data[3].buid_value = 0xffffffff
buc_info_ptr->buid_data[3].buid_Sptr = 0x0
buc_info_ptr->mem_alloc1 = 0x8000
buc_info_ptr->mem_addr1 = 0xff8000
buc_info_ptr->mem_alloc2 = 0x0
buc_info_ptr->mem_addr2 = 0x0
buc_info_ptr->vpd_rom_width = 0xffffffff
buc_info_ptr->cfg_addr_inc = 0x0
buc_info_ptr->device_id_reg = 0x2040
buc_info_ptr->aux_info_offset = 0x0
buc_info_ptr->feature_rom_code = 0x0
buc_info_ptr->IOCC_flag = 0x0
buc_info_ptr->location[0] = 0x30
buc_info_ptr->location[1] = 0x30
buc_info_ptr->location[2] = 0x30
buc_info_ptr->location[3] = 0x30
buc_info_ptr->num_of_structs = 0x3
buc_info_ptr->index = 0x2
buc_info_ptr->struct_size = 0x70
buc_info_ptr->bsrr_offset = 0x0
buc_info_ptr->bsrr_mask = 0x0
buc_info_ptr->bscr_value = 0x0
buc_info_ptr->cfg_status = 0x2
buc_info_ptr->device_type = 0x5
buc_info_ptr->num_of_buids = 0x2
buc_info_ptr->buid_data[0].buid_value = 0x100
buc_info_ptr->buid_data[0].buid_Sptr = 0x80000000
buc_info_ptr->buid_data[1].buid_value = 0x10100
buc_info_ptr->buid_data[1].buid_Sptr = 0xc0000000
buc_info_ptr->buid_data[2].buid_value = 0xffffffff
buc_info_ptr->buid_data[2].buid_Sptr = 0x0
buc_info_ptr->buid_data[3].buid_value = 0xffffffff
buc_info_ptr->buid_data[3].buid_Sptr = 0x0
buc_info_ptr->mem_alloc1 = 0x0
buc_info_ptr->mem_addr1 = 0x0
buc_info_ptr->mem_alloc2 = 0x0
buc_info_ptr->mem_addr2 = 0x0
buc_info_ptr->vpd_rom_width = 0xffffffff
buc_info_ptr->cfg_addr_inc = 0x0
buc_info_ptr->device_id_reg = 0x2020
buc_info_ptr->aux_info_offset = 0x0
buc_info_ptr->feature_rom_code = 0x0
buc_info_ptr->IOCC_flag = 0x1
buc_info_ptr->location[0] = 0x30
buc_info_ptr->location[1] = 0x30
buc_info_ptr->location[2] = 0x31
buc_info_ptr->location[3] = 0x30
sys_info->nvram_size = 0x0
sys_info->nvram_addr = 0x0
sys_info->todr_addr = 0x0
sys_info->architecture = 0x2
sys_info->implementation = 0x3
sys_info->pkg_descriptor="MOT3F00"
proc_info->num_of_structs = 0x1
proc_info->index = 0x0
proc_info->struct_size = 0xc8
proc_info->per_buc_info_offset = 0x3c749f4
proc_info->proc_int_area = 0x0
proc_info->proc_int_area_size = 0x0
proc_info->processor_present = 0x1
proc_info->test_run = 0xd5
proc_info->test_stat = 0x0
proc_info->link = 0x0
proc_info->link_address = 0x0
proc_info->phys_id = 0x0
proc_info->priv_lck_cnt = 0x0
proc_info->prob_lck_cnt = 0x0
proc_info->architecture = 0x2
proc_info->implementation = 0x10
proc_info->width = 0x20
proc_info->cache_attrib = 0x1
proc_info->icache_size = 0x8000
proc_info->dcache_size = 0x8000
proc_info->icache_asc = 0x4
proc_info->dcache_asc = 0x4
proc_info->tlb_attrib = 0x1
proc_info->itlb_size = 0x80
proc_info->dtlb_size = 0x80
proc_info->itlb_asc = 0x2
proc_info->dtlb_asc = 0x2
proc_info->slb_attrib = 0x0
proc_info->islb_size = 0x0
proc_info->dslb_size = 0x0
proc_info->islb_asc = 0x0
proc_info->dslb_asc = 0x0
proc_info->rtc_type = 0x2
proc_info->rtcXint = 0x0
proc_info->rtcXfrac = 0x0
proc_info->tbCfreq_HZ = 0x7f2815
proc_info->busCfreq_HZ = 0x0
proc_info->version = 0x50000
proc_info->L2_cache_size = 0x0
proc_info->L2_cache_asc = 0x0
proc_info->coherency_size = 0x20
proc_info->resv_size = 0x20
proc_info->icache_block = 0x20
proc_info->dcache_block = 0x20
proc_info->icache_line = 0x20
proc_info->dcache_line = 0x20
proc_info->proc_descriptor = "PowerPC_604"
l2_data->num_of_structs = 0x1
l2_data->index = 0x0
l2_data->struct_size = 0xc0
l2_data->shared_L2_cache = 0x0
l2_data->using_resource_offset = 0xa64
l2_data->mode = 0x0
l2_data->installed_size = 0x0
l2_data->configured_size = 0x0
l2_data->size[0] = 0x0
l2_data->type[0] = 0x30
l2_data->type[1] = 0x30
l2_data->adapter_present = 0x0
l2_data->adapter_bad = 0x0
mem_data[i].num_of_structs = 0x6
mem_data[i].struct_size = 0x34
mem_data[i].card_or_SIMM_size = 0x3e
mem_data[i].state = 0x1
mem_data[i].num_of_bad_simms = 0x0
mem_data[i].card_or_simm_indicator = 0x1
mem_data[i].EC_level = 0x0
mem_data[i].PD_bits = 0x0
mem_data[i].location [0][0] = 0x30
mem_data[i].location [0][1] = 0x30
mem_data[i].location [0][2] = 0x30
mem_data[i].num_of_structs = 0x6
mem_data[i].struct_size = 0x34
mem_data[i].card_or_SIMM_size = 0x0
mem_data[i].state = 0x0
mem_data[i].num_of_bad_simms = 0x0
mem_data[i].card_or_simm_indicator = 0x1
mem_data[i].EC_level = 0x0
mem_data[i].PD_bits = 0x0
mem_data[i].location [0][0] = 0x30
mem_data[i].location [0][1] = 0x30
mem_data[i].location [0][2] = 0x30
mem_data[i].location [0][3] = 0x42
mem_data[i].num_of_structs = 0x6
mem_data[i].struct_size = 0x34
mem_data[i].card_or_SIMM_size = 0x0
mem_data[i].state = 0x0
mem_data[i].num_of_bad_simms = 0x0
mem_data[i].card_or_simm_indicator = 0x1
mem_data[i].EC_level = 0x0
mem_data[i].PD_bits = 0x0
mem_data[i].location [0][0] = 0x30
mem_data[i].location [0][1] = 0x30
mem_data[i].location [0][2] = 0x30
mem_data[i].location [0][3] = 0x43
mem_data[i].num_of_structs = 0x6
mem_data[i].struct_size = 0x34
mem_data[i].card_or_SIMM_size = 0x0
mem_data[i].state = 0x0
mem_data[i].num_of_bad_simms = 0x0
mem_data[i].card_or_simm_indicator = 0x1
mem_data[i].EC_level = 0x0
mem_data[i].PD_bits = 0x0
mem_data[i].location [0][0] = 0x30
mem_data[i].location [0][1] = 0x30
mem_data[i].location [0][2] = 0x30
mem_data[i].location [0][3] = 0x44
mem_data[i].num_of_structs = 0x6
mem_data[i].struct_size = 0x34
mem_data[i].card_or_SIMM_size = 0x0
mem_data[i].state = 0x0
mem_data[i].num_of_bad_simms = 0x0
mem_data[i].card_or_simm_indicator = 0x1
mem_data[i].EC_level = 0x0
mem_data[i].PD_bits = 0x0
mem_data[i].location [0][0] = 0x30
mem_data[i].location [0][1] = 0x30
mem_data[i].location [0][2] = 0x30
mem_data[i].location [0][3] = 0x45
mem_data[i].num_of_structs = 0x6
mem_data[i].struct_size = 0x34
mem_data[i].card_or_SIMM_size = 0x0
mem_data[i].state = 0x0
mem_data[i].num_of_bad_simms = 0x0
mem_data[i].card_or_simm_indicator = 0x1
mem_data[i].EC_level = 0x0
mem_data[i].PD_bits = 0x0
mem_data[i].location [0][0] = 0x30
mem_data[i].location [0][1] = 0x30
mem_data[i].location [0][2] = 0x30
mem_data[i].location [0][3] = 0x46
user_info->user_data_len = 0x50
user_info->user_id_offset = 0x9390
user_info->next_offset = 0x0
mot_data->company = "Motorola Computer Group"
mot_data->board_model = 0x6
mot_data->board_revision = 0x42
mot_data->ethernet_na = 45 55 55 55 45 55
LED(MOTLED_RELOCATING_KERNEL)=0x13e0
+ swcons -c

Saving Base Customize Data to boot disk
Starting the sync daemon
Starting the error daemon
System initialization completed.
Starting Multi-user Initialization
 Performing auto-varyon of Volume Groups
 Activating all paging spaces
swapon: Paging device /dev/hd6 activated.
 Performing all automatic mounts
mount: 1831-010 server a2231s01 not responding: RPC: 1832-018 Port mapper failure - RPC: 1832-006 Unable to send
mount: backgrounding
a2231s01:/home
Multi-user initialization completed
Checking for srcmstr active...complete
Starting tcpip daemons:
0513-059 The syslogd Subsystem has been started. Subsystem PID is 3434.
0513-059 The sendmail Subsystem has been started. Subsystem PID is 5232.
0513-059 The portmap Subsystem has been started. Subsystem PID is 5494.
0513-059 The inetd Subsystem has been started. Subsystem PID is 5756.
0513-059 The snmpd Subsystem has been started. Subsystem PID is 6018.
0513-059 The dpid2 Subsystem has been started. Subsystem PID is 6280.
0513-059 The muxatmd Subsystem has been started. Subsystem PID is 6542.
0513-059 The fibred Subsystem has been started. Subsystem PID is 6804.
vmtune:  current values:
  -p       -P        -r          -R         -f       -F       -N        -W
minperm  maxperm  minpgahead maxpgahead  minfree  maxfree  pd_npages maxrandwrt
   2968    11872       2          8        115      123     524288        0

  -M       -w       -k       -c         -b          -B          -u
maxpin   npswarn  npskill  numclust  numfsbufs   hd_pbuf_cnt  lvm_bufcnt
  12692     1536      384        0       93           64           9

number of valid memory pages = 15864    maxperm=74.8% of real memory
maximum pinable=80.0% of real memory    minperm=18.7% of real memory
number of file memory pages = 1445      numperm=9.1% of real memory

The next step would be to get it working under QEMU.
Under QEMU it gets pretty far, it does find the PCI- and the ISA buses and even the SCSI host.

Unfortunately it doesn't find SCSI disks. Here it is tricky, it may be a problem with the interrupt routing, or DMA or SCSI host emulation.

/Stay tuned

My new toys: Motorola PowerStack II

The second gift from Jochen is a Powerstack II mainboard with an AT power supply unit and a SCSI disk. The SCSI disk has "AIX" written on it, which looks promising, but Jochen doesn't remember if it was really installed, or just planned.

The board has Serial/Parallel/Ethernet/SCSI and even a couple of unsoldered IDE connectors.
The boot log shows it has a Firmworks based Open Firmware:

WARNING: NVRAM Header Test Failed - Auto Initializing
Starting real time clock...
screen not found.
Can't open input device.
Keyboard not present.  Using com1 for input and output.
, Serial #0, 64 MB memory
Power Firmware(TM) by FirmWorks , Built  Thu Jun 4 10:20:43 MST 1998
Copyright (c) 1995-1996 FirmWorks.  All Rights Reserved.
PowerPC Open Firmware
Version 1.2 RM11   Thu Jun 4 10:20:43 MST 1998
Copyright Motorola 1995-96, All Rights Reserved
Copyright FirmWorks 1995-96, All Rights Reserved

 CPU . . . . . . . . . . . . . . . . . . . . . . . . . . . =PowerPC,604e
 MicroProcessor Internal Clock Speed (MHZ) . . . . . . . . =300
 MicroProcessor External Clock Speed (MHZ) . . . . . . . . =67
 PCI Bus Clock Speed (MHZ) . . . . . . . . . . . . . . . . =33
 Local Memory Size . . . . . . . . . . . . . . . . . . . . =4000000 (64 MB)
 Memory Type . . . . . . . . . . . . . . . . . . . . . . . =EDO
 Memory Error Checking . . . . . . . . . . . . . . . . . . =ECC
 Memory Speed. . . . . . . . . . . . . . . . . . . . . . . =50 NS
 L2 Cache Size . . . . . . . . . . . . . . . . . . . . . . =256KB
 L2 Cache Type . . . . . . . . . . . . . . . . . . . . . . =Asynchronous
 L2 Cache Parity . . . . . . . . . . . . . . . . . . . . . =Disabled
 Configuration Checksum. . . . . . . . . . . . . . . . . . =Failed

Then it gets to a windowed menu interface (which doesn't look like the typical OFW at all), but under "Administrative options" it's possible to choose "Invoke the Command Line Prompt", which gives the famous "ok" prompt.

AIX starts booting from the SCSI disk:

Trying..., fdisk0 Recalibrate failed.  The floppy drive is either missing,
improperly connected, or defective.
Failed
Trying..., hdisk0 Booting
Please wait while the system is booting
Boot device: /pci/scsi@2/disk@6,0  File and args:
   

 ******* Please define the System Console. *******

 Type a 1 and press Enter to use this terminal as the
  system console.

cvga0
+ swcons -c

Saving Base Customize Data to boot disk
Starting the sync daemon
Starting the error daemon
System initialization completed.
Starting Multi-user Initialization
 Performing auto-varyon of Volume Groups
 Activating all paging spaces
swapon: Paging device /dev/hd6 activated.
 Performing all automatic mounts

And here it hangs. Probably it tries to perform a NFS mount, which I don't have.
Anyways it's much further than QEMU currently gets, so it's definitely can be used as a reference.

I don't have a UW-SCSI cdrom drive to boot from the Powerstack II  media. But it can be netbooted via tftp.

Surprisingly booting the Solaris/PPC did not work out. The floppy is not recognized, tried to netboot SOLARIS.ELF from the cd got an interesting error:

Rebooting with command: boot /pci/ethernet@4:172.22.0.20,SOLARIS.ELF,172.22.134.1
Boot device: /pci/ethernet@4:172.22.0.250,SOLARIS.ELF,172.22.134.51  File and args:
Trying to get Internet/Ethernet Address ...
       Contact your system administrator to see if
       a Boot Host and network is setup correctly.

so, obviously after switching to the Little-Endian mode the Motorola network driver doesn't work anymore. Looks like in 1998 netbooting Solaris and Windows NT was not relevant for Motorola anymore, otherwise it would have been tested.

Overall it looks like Motorola did heavily modify the OFW. For instance, there are no hidden words. Which is nice. It should be possible to peek if it has any quirks in creation of the residual data. Or better to say it could have been. It is all a one single quirk, there is no residual data.

Initially I thought that this would be a perfect firmware which would boot both PReP images and the later OFW-compatible ones. But alas. After poking around, I googled and found a couple of mails on the NetBSD mailing list stating that:
 1. The firmware doesn't provide any residual data
 2. The firmware doesn't have the PCI, DMA and interrupt mapping properties in the device tree.
Looking at the code I see that the first point is clearly caused by the second one. In the OFW the residual data is generated from the device tree. The code was not removed, but Motorola forgot to add the properties. 

Which makes it a worst possible firmware.

But still it can boot the AIX from the supplied SCSI disk. This explains at least one reason for a custom AIX: the Motorola version should be able to live without the residual data.

Probably the developers were in a rush, so instead of fixing the firmware properties, they just added a hack to the OS. Maybe the OS department had more resources than the firmware one, or maybe the developers who were able to do Forth, were on vacation or fired.

The result is ugly, but I think every software developer has done something similar at least once.

Anyway now I have a sort of reference machine which can sort of boot AIX.

P.S. And by the way, if you wonder why I keep writing  "Powerstack II Utah" instead of just "Powerstack II". It turned out multiple machines called "Powerstack II" were produced. And indeed they are incompatible. More gory details in the Linux kernel sources.

My new toys: Motorola Powerstack I

Jochen Kunz sent me two Motorola PowerStack toys. Thank you very much, Jochen. Now I should have the reference machines which I can use for fixing QEMU.

The first one is a classic PowerStack I machine, which looks pretty cool and has some kind of proprietary firmware (PPC1Bug).  To connect it via a serial line I had to use pretty much all the cables and adapters I have: short 9F-9F cable, 9M-25F adapter, 25F-25F cable, 25M-9F adapter.
On the desktop Linux side I use GNU screen on a serial line. Found this feature just a few days ago. For those who missed it too, that's how it gets attached to a running screen:

screen -X screen /dev/ttyS0  # note 2 screens, that's not typo

That's what it tells on powering on:

Copyright Motorola Inc. 1988 - 1995, All Rights Reserved

PPC1 Debugger/Diagnostics Release Version 1.8 - 10/04/95
COLD Start

Local Memory Found =02000000 (&33554432)

WARNING: Board Configuration Data Failure

MPU Clock Speed =100Mhz
WARNING: Keyboard not connected

Initializing System Memory (DRAM)...

System Memory: 32MB, Parity Enabled (Parity-Memory Detected)
L2Cache:       NONE, Parity NOT Enabled


SelfTest/Boots about to Begin... Press <BREAK> at anytime to Abort ALL

SelfTest about to Begin... Press <ESC> to Bypass, <SPC> to Continue

RAM      ADR: Addressability......................... Running ---> PASSED
PC16550  REGA: Register Access....................... Running ---> PASSED

PC16550  IRQ: Interrupt.............................. Running ---> PASSED
PC16550  BAUD: Baud Rate............................. Running ---> PASSED

PC16550  LPBK: Internal Loopback..................... Running ---> PASSED
Z8536    CNT: Counter................................ Running ---> PASSED

Z8536    LNK: Linked Counter......................... Running ---> PASSED
Z8536    IRQ: Interrupt.............................. Running ---> PASSED

Z8536    REG: Register............................... Running ---> PASSED
SCC      ACCESS: Device/Register Access.............. Running ---> PASSED

SCC      IRQ: Interrupt Request...................... Running ---> PASSED
PAR87303 REG: PC87303 Parallel Port's Register/Data.. Running ---> PASSED

DEC21040 REGA: PCI Register Access................... Running ---> PASSED
DEC21040 XREGA: Extended PCI Register Access......... Running ---> PASSED

DEC21040 SPACK: Single Packet Xmit/Recv.............. Running ---> PASSED
DEC21040 ILR: Interrupt Line Register Access......... Running ---> PASSED

DEC21040 ERREN: ERREN and SERREN Bit Toggle.......... Running ---> PASSED
DEC21040 IOR: I/O Resource Register Access........... Running ---> PASSED

DEC21040 CINIT: Chip Initialization.................. Running ---> PASSED
NCR      PCI: NCR 53c8xx PCI Access.................. Running ---> PASSED

NCR      ACC1: NCR 53c8xx Device Access.............. Running ---> PASSED
NCR      ACC2: NCR 53c8xx Register Access............ Running ---> PASSED

NCR      SFIFO: NCR 53c8xx SCSI FIFO................. Running ---> PASSED
NCR      DFIFO: NCR 53c8xx DMA FIFO.................. Running ---> PASSED

NCR      IRQ: NCR 53c8xx Interrupts.................. Running ---> PASSED
NCR      SCRIPTS: NCR 53c8xx SCRIPTs Processor....... Running ---> PASSED

I82378   REG: i82378 Register Access................. Running ---> PASSED
I82378   IRQ: Interrupt Request...................... Running ---> PASSED

AutoBoot about to Begin... Press <ESC> to Bypass, <SPC> to Continue

NetBoot about to Begin... Press <ESC> to Bypass, <SPC> to Continue

1) Continue System Start Up
2) Select Alternate Boot Device
3) Go to System Debugger
4) Initiate Service Call
5) Display System Test Errors
6) Dump Memory to Tape
Enter Menu #:

It doesn't have anything on its hard drive, so the only reasonable option here is 3):

PPC1-Diag>ioi
I/O Inquiry Status:
CLUN  DLUN  CNTRL-TYPE  DADDR  DTYPE  RM  Inquiry-Data
  0     0  NCR53C825   0      $00    N   SEAGATE  ST31230W         0456
  0    50  NCR53C825   5      $05    Y   TOSHIBA  CD-ROM XM-4101TA 1084
  1     0  PC8477      0      $00    Y   <None> 

Tried all boot disks I have.

+ Boots Solaris 2.5.1/PPC floppy, which provides some very limited Open Firmware (not even sure it's based on the Firmworks OFW). After booting the floppy it's possible to boot Solaris from a CD. Nice to have, but not my toy of the choice: it works in a little endian mode, which currently doesn't work under QEMU/PReP, and hardly has any software. But if one day all the other OSes emulated I may get back to it.

- Unsurprisingly doesn't boot from any IBM AIX CDs. Already heard that AIX is quite picky about the hardware, was just curious if it gives any error message. It doesn't.

- Surprisingly doesn't boot from the two Motorla AIX CDs I have:
    "AOS1_3__RM02" (aka AIX v4.1.4 for Motorola PowerStack II)
    "AOS1_4__RM03" (aka AIX v4.1.4r4 for Motorola PowerStack II)
So, obviously the PowerStack II AIX is not compatible with PowerStack I.

* Haven't tried booting Windows NT on it. There is a report in google groups that NT flashes another firmware which can only boot NT and it's not possible to get back to PPC1Bug. On top of that, NT is little-endian, just like Solaris 2.5.1/PPC, so all the considerations from the above apply here too.

The good news: it has  an i82378 PCI controller and a NCR53C825 SCSI, which is quite close to what QEMU/PReP/40p target currently emulates.
The bad news: unless I find a boot disk for AIX for Motorola PowerStack I, this machine can not be used for debugging AIX.

The next weekend I'll write about the second toy.

/Stay tuned

Does anybody use recent qemu versions?

After a long break I tried a current qemu git/master today. What can I say? The SPARC-related stuff is broken in a few places since a few months. SCSI signals an error for the "Inquiry" command, Leon3 test hangs, sparc64 is broken too. Bisecting is tricky though, because at least sparc64 seems to broken in different ways in subsequent commits.

Don't have time to fix it myself, but will send a couple of bug reports to the mailing list. Meanwhile, what is your favorite qemu version?

Update: meanwhile (7/Feb/2012) SCSI and Leon3 issues are fixed, thanks to Thomas Higdon and Fabien Chouteau.

Bug in NetBSD 1.6 - 3.1 emulation

Was going to write that I couldn't imagine what did the NetBSD guys smoke during 5 years between the versions 1.6 and 3.1 inclusively. The NCR 53c9x SCSI chip (known as "esp" in SPARC and PPC machines) was not so uncommon back then. How could they introduce an instability and didn't notice it ?!?

The code from NetBSD 1.5.3:

NCRDMA_SETUP(sc, &sc->sc_cmdp, &sc->sc_cmdlen, 0, &dmasize);
//...
NCRCMD(sc, NCRCMD_SELATN | NCRCMD_DMA);
NCRDMA_GO(sc);

The code from NetBSD 1.6-3.1:

NCRCMD(sc, NCRCMD_SELATN | NCRCMD_DMA);
NCRDMA_SETUP(sc, &sc->sc_cmdp, &sc->sc_cmdlen, 0, &dmasize);
NCRDMA_GO(sc);

The code from NetBSD 4:

NCRDMA_SETUP(sc, &sc->sc_cmdp, &sc->sc_cmdlen, 0, &dmasize);
//...
NCRCMD(sc, NCRCMD_SELATN | NCRCMD_DMA);
NCRDMA_GO(sc);

See the difference? In the versions 1.6-3.1 the command is executed before the DMA is set up, so the SCSI controller may not get the command using DMA.

And then I googled for the expected bug reports and found none. Why could it work on the real hardware? Maybe some latency: if the latency of a SCSI controller was larger than a DMA controller, it might work? Maybe concurrency: if some other driver (for instance Ethernet) prepared DMA for itself, the SCSI driver could steal it? Maybe DMA didn't work for these versions at all and after the first attempt they switched to the PIO mode.

Another week another SCSI bug

Fixed Solaris 2.6+ boot which I accidentally broke last week. It's not that my Solaris 2.3 dma/irq fix was wrong, but the fix unleashed a counterpart interrupts handling bug in esp controller.

Too bad that no one reported it earlier. I wouldn't have to hack till midnight now. ;-) And thanks to VooDoo_UzH_ for reporting it.

Trying to reach 1993

Trying to boot yet older Solaris/SunOS version: Solaris 2.3. According to the Wikipedia, it's the first one which supported SPARCStation-5. You may wanna ask "what is about SunOS 4.1.4 (Solaris 1.1.2), it can be booted, and it must be the older?". No, it's not. Looks like Solaris 2.x and SunOS 4.x were developed independently: 1.1.2 was released in November 1994, and 2.3 was released in November 1993. Is a bit misleading. But explains why 2.3 has problems with the esp scsi controller while 1.1.2 doesn't. And the both systems are so old that the kadb debugger can't set deferred breakpoints. Anyway, the current status of Solaris 2.3:

ok boot disk1:d -vs
Boot device: /iommu/sbus/espdma@5,8400000/esp@5,8800000/sd@1,0:d  File and args: -vs
Size: 719688+166144+108728 Bytes
SunOS Release 5.3 Version Generic [UNIX(R) System V Release 4.0]
Copyright (c) 1983-1993, Sun Microsystems, Inc.
vac: enabled in write through mode
cpu0: FMI,MB86907 (mid 0 impl 0x0 ver 0x4 clock 1075 MHz)
mem = 65536K (0x4000000)
avail mem = 58142720
Ethernet address = 52:54:0:12:34:56
root nexus = SUNW,SPARCstation-5
iommu0 at root: obio 0x10000000
sbus0 at iommu0: obio 0x10001000
espdma0 at sbus0: SBus slot 5 0x8400000
esp0 at espdma0: SBus slot 5 0x8800000 sparc ipl 4
        polled command timeout
esp:            State=CLEARING Last State=CLEARING
esp:            Latched stat=0x97 intr=0x8 fifo 0x0
esp:            last msg out: NO-OP; last msg in: COMMAND COMPLETE
esp:            DMA csr=0xa4240030
esp:            addr=fc00300a dmacnt=8000 last=fc003008 last_cnt=30
esp:            Cmd dump for Target 1 Lun 0:
esp:            cdblen=6, cdb=[ 0x12 0x0 0x0 0x0 0x30 0x0 ]; Status=0x0
esp:            pkt_state=0x1f pkt_flags=0xb pkt_statistics=0x0
esp:            cmd_flags=0x1422 cmd_timeout=60

another week, another qemu bug

There are very few qemu/sparc modules out there which I haven't had to touch. Since I've started I founded/fixed bugs in: irq, esp, cpu, esp, esp, cpu, scsi-disk, cpu,  scsi-disk, fdd, tcx, mmu, slavio, mmu. Today this list is extended with (sparc32_)dma.

Fixed a bug in dma which produced spurious interrupts and incomplete reads/writes. Will submit the patch later on this week.

Spurious interrupts

Previously qemu dropped interrupts on disabling them. The real hardware doesn't do it. Which means, that lots of interrupts were dropped, including the spurious ones. But the real ones were dropped too, that's why the system timer was ticking so slow.

The question is where the spurious  IRQs are coming from: it's not only the ESP which produces them, under Solaris 8 & 9 there are lots of complains about spurious timer interrupts, and they both seem to crash due to a buffer overflow during processing of a (possibly spurious) LE interrupt.

NetBSD 1.3.3 boot doesn't crash with my wrong irq patch. The patch makes qemu drop interrupts, including the spurious ones.

All in all it looked like there is one global problem with interrupts processing. Until now. But now it looks like there is not one source of spurious interrupts, but many.

• esp doesn't seem to produce spurious interrupts under Solaris while reading.
• I've found a bug in the slavio timer which produces spurious interrupts.
• NetBSD may be crashing due to another issue: I don't have a disk which I could boot under OBP. It is possible that OpenBIOS is not compatible with the older NetBSD versions. Update: This is the reason for NetBSD crashing, Michael Kostylev confirmed it.

Another small improvement in SCSI emulation

The message

Error: Inquiry (STANDARD) buffer size 5 is less than 36 (TODO: only 5 required)

was quite annoying, so I attacked it. I used to have a hack which explicitly implemented
inquiry with the allocation size length == 5.
But it turned out that the clean fix is quite trivial, the specification says "if the allocation length of the command descriptor block (CDB) is too small to transfer all of the parameters, the additional length shall not be adjusted to reflect the truncation", so the clean fix for this problem is not longer than the code telling about "TODO". :) Will send the patch upstream. Now probe-scsi in OBP looks really nice:

ok probe-scsi
Target 0
Unit 0 Disk QEMU QEMU HARDDISK 0.11
Target 1
Unit 0 Disk QEMU QEMU HARDDISK 0.11
Target 2
Unit 0 Removable Read Only device QEMU QEMU CD-ROM 0.11
ok

A little improvement of SCSI disk emulation

NetBSD complained:

sd3: mode sense (4) returned nonsense; using fictitious geometry

SunOS 4.1.4 complains:

sd3: non-CCS device found at target 0 lun 0 on esp0

I was hoping that this is the same bug, so I tried to investigate. It's always easier to work when the sources are available. It turned out that NetBSD expects a block descriptor for mode pages, which wasn't implemented in qemu, and is probably mandatory for SCSI-2 disks. The specs are unclear: they don't explicitly say "optional" or "mandatory" in chapters 9.1.2 and 9.3.3.

Anyway, I implemented the block descriptor, and the NetBSD bug is gone. But this was in vain: the SunOS bug is still there, the Solaris bootblk problem is also un-affected.

Will send the patch upstream though. A small improvement is still an improvement.

Third bug in SCSI layer (esp) fixed

Up to now "select without attention" was handled the same way as "select with attention". According to NCR53C9X documentation, select without ATN sends the CDB (Command Descriptor Block) directly, whereas select with ATN sends one message phase byte followed by 6, 10, or 12 command phase bytes. This one byte was shifting CDB and producing invalid commands. After fixing this bug scsi probe looks like this:

ok probe-scsi
Target 0
Unit 0 Disk
Target 2
Unit 0 Removable Read Only device
ok

It still doesn't show all the target properties, but it doesn't matter. The next stop is booting.

second (and third) bugs in SCSI (esp) emulation

It looks like "Message Accepted" shouldn't write a response. At least ESP_RFLAGS must definetely be 0.

After I fixed the bug, OBP got one step further. Now it sees the targets:

ok probe-scsi
Target 0
scsi-disk: Unsupported command length, command 60
scsi-disk: Unsupported command length, command 60
scsi-disk: Unsupported command length, command c0
scsi-disk: Unsupported command length, command c0
scsi-disk: Unsupported command length, command e0
scsi-disk: Unsupported command length, command e0
Target 2
scsi-disk: Unsupported command length, command 60
scsi-disk: Unsupported command length, command 60
scsi-disk: Unsupported command length, command c0
scsi-disk: Unsupported command length, command c0
scsi-disk: Unsupported command length, command e0
scsi-disk: Unsupported command length, command e0
ok

Next stop is inquiring targets parameters.

got past scsi-controller initialization

got past scsi-controller initialization. The next stop is disks probing:

ok probe-scsi
Extra scsi data. Fatal error.Extra scsi data. Fatal error.
ok