README: Add travis build status badge #1
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Samuel Ortiz <[email protected]>
rbradford
approved these changes
May 14, 2019
sameo
referenced
this pull request
in sameo/cloud-hypervisor-1
Sep 19, 2019
With the API server socket option, we want to support new UX patterns: 1) The CLI model: The user entirely configures a guest from the command line and does not expect to be able to control it dynamically through the API socket. This is the current model. 2) The API control model: The user does not configure anything from the command line, and calls the cloud-hypervisor binary with the API socket path command line option only. Everything will be done from the API socket, including the guest initial configuration and creation. 3) The hybrid model: The user wants to quickly create and start a guest and then asynchronously control it from the API socket. The initial configuration setup is done through the command line and further control go through the API socket. In this model, we need to make sure that the guest configuration is valid. We don't want to support invalid configurations that could implicitly be overwritten through the API socket. In order to support the new models (#2 and #3) we need to be able to tell if an empty guest configuration was passed through the command line or not. If it is empty, then we go with model #2. If it is not empty, we either go with #1 or #3. One way to check if a guest configuration is empty or not is by logically grouping the command line options into a VM config group and a VMM config one. If the VM config group is empty, then the user is not trying to configure a guest. Signed-off-by: Samuel Ortiz <[email protected]>
sameo
referenced
this pull request
in sameo/cloud-hypervisor-1
Sep 19, 2019
With the API server socket option, we want to support new UX patterns: 1) The CLI model: The user entirely configures a guest from the command line and does not expect to be able to control it dynamically through the API socket. This is the current model. 2) The API control model: The user does not configure anything from the command line, and calls the cloud-hypervisor binary with the API socket path command line option only. Everything will be done from the API socket, including the guest initial configuration and creation. 3) The hybrid model: The user wants to quickly create and start a guest and then asynchronously control it from the API socket. The initial configuration setup is done through the command line and further control go through the API socket. In this model, we need to make sure that the guest configuration is valid. We don't want to support invalid configurations that could implicitly be overwritten through the API socket. In order to support the new models (#2 and #3) we need to be able to tell if an empty guest configuration was passed through the command line or not. If it is empty, then we go with model #2. If it is not empty, we either go with #1 or #3. One way to check if a guest configuration is empty or not is by logically grouping the command line options into a VM config group and a VMM config one. If the VM config group is empty, then the user is not trying to configure a guest. Signed-off-by: Samuel Ortiz <[email protected]>
arafatms
pushed a commit
to arafatms/cloud-hypervisor
that referenced
this pull request
Aug 3, 2021
… I/O tracking backend like SPDK required to know how many virt queues to be handled before gets VHOST_USER_SET_INFLIGHT_FD message. fix dpdk core dump while processing vhost_user_set_inflight_fd: #0 0x00007fffef47c347 in vhost_user_set_inflight_fd (pdev=0x7fffe2895998, msg=0x7fffe28956f0, main_fd=545) at ../lib/librte_vhost/vhost_user.c:1570 cloud-hypervisor#1 0x00007fffef47e7b9 in vhost_user_msg_handler (vid=0, fd=545) at ../lib/librte_vhost/vhost_user.c:2735 cloud-hypervisor#2 0x00007fffef46bac0 in vhost_user_read_cb (connfd=545, dat=0x7fffdc0008c0, remove=0x7fffe2895a64) at ../lib/librte_vhost/socket.c:309 cloud-hypervisor#3 0x00007fffef45b3f6 in fdset_event_dispatch (arg=0x7fffef6dc2e0 <vhost_user+8192>) at ../lib/librte_vhost/fd_man.c:286 cloud-hypervisor#4 0x00007ffff09926f3 in rte_thread_init (arg=0x15ee180) at ../lib/librte_eal/common/eal_common_thread.c:175 Signed-off-by: Arafatms <[email protected]>
arafatms
added a commit
to arafatms/cloud-hypervisor
that referenced
this pull request
Aug 3, 2021
… I/O tracking backend like SPDK required to know how many virt queues to be handled before gets VHOST_USER_SET_INFLIGHT_FD message. fix dpdk core dump while processing vhost_user_set_inflight_fd: #0 0x00007fffef47c347 in vhost_user_set_inflight_fd (pdev=0x7fffe2895998, msg=0x7fffe28956f0, main_fd=545) at ../lib/librte_vhost/vhost_user.c:1570 cloud-hypervisor#1 0x00007fffef47e7b9 in vhost_user_msg_handler (vid=0, fd=545) at ../lib/librte_vhost/vhost_user.c:2735 cloud-hypervisor#2 0x00007fffef46bac0 in vhost_user_read_cb (connfd=545, dat=0x7fffdc0008c0, remove=0x7fffe2895a64) at ../lib/librte_vhost/socket.c:309 cloud-hypervisor#3 0x00007fffef45b3f6 in fdset_event_dispatch (arg=0x7fffef6dc2e0 <vhost_user+8192>) at ../lib/librte_vhost/fd_man.c:286 cloud-hypervisor#4 0x00007ffff09926f3 in rte_thread_init (arg=0x15ee180) at ../lib/librte_eal/common/eal_common_thread.c:175 Signed-off-by: Arafatms <[email protected]>
arafatms
added a commit
to arafatms/cloud-hypervisor
that referenced
this pull request
Aug 4, 2021
…I/O tracking
backend like SPDK required to know how many virt queues to be handled
before gets VHOST_USER_SET_INFLIGHT_FD message.
fix dpdk core dump while processing vhost_user_set_inflight_fd:
#0 0x00007fffef47c347 in vhost_user_set_inflight_fd (pdev=0x7fffe2895998, msg=0x7fffe28956f0, main_fd=545) at ../lib/librte_vhost/vhost_user.c:1570
cloud-hypervisor#1 0x00007fffef47e7b9 in vhost_user_msg_handler (vid=0, fd=545) at ../lib/librte_vhost/vhost_user.c:2735
cloud-hypervisor#2 0x00007fffef46bac0 in vhost_user_read_cb (connfd=545, dat=0x7fffdc0008c0, remove=0x7fffe2895a64) at ../lib/librte_vhost/socket.c:309
cloud-hypervisor#3 0x00007fffef45b3f6 in fdset_event_dispatch (arg=0x7fffef6dc2e0 <vhost_user+8192>) at ../lib/librte_vhost/fd_man.c:286
cloud-hypervisor#4 0x00007ffff09926f3 in rte_thread_init (arg=0x15ee180) at ../lib/librte_eal/common/eal_common_thread.c:175
Signed-off-by: Arafatms <[email protected]>
sboeuf
pushed a commit
that referenced
this pull request
Aug 4, 2021
…I/O tracking
backend like SPDK required to know how many virt queues to be handled
before gets VHOST_USER_SET_INFLIGHT_FD message.
fix dpdk core dump while processing vhost_user_set_inflight_fd:
#0 0x00007fffef47c347 in vhost_user_set_inflight_fd (pdev=0x7fffe2895998, msg=0x7fffe28956f0, main_fd=545) at ../lib/librte_vhost/vhost_user.c:1570
#1 0x00007fffef47e7b9 in vhost_user_msg_handler (vid=0, fd=545) at ../lib/librte_vhost/vhost_user.c:2735
#2 0x00007fffef46bac0 in vhost_user_read_cb (connfd=545, dat=0x7fffdc0008c0, remove=0x7fffe2895a64) at ../lib/librte_vhost/socket.c:309
#3 0x00007fffef45b3f6 in fdset_event_dispatch (arg=0x7fffef6dc2e0 <vhost_user+8192>) at ../lib/librte_vhost/fd_man.c:286
#4 0x00007ffff09926f3 in rte_thread_init (arg=0x15ee180) at ../lib/librte_eal/common/eal_common_thread.c:175
Signed-off-by: Arafatms <[email protected]>
sboeuf
pushed a commit
that referenced
this pull request
Jan 4, 2022
This updated tree now includes a fix for virtio-net regression that was present in 5.14: commit 732b74d647048668f0f8dc0c848f0746c69e2e2f Author: Xuan Zhuo <[email protected]> Date: Sat Oct 9 05:17:53 2021 -0400 virtio-net: fix for skb_over_panic inside big mode commit 126285651b7f ("Merge ra.kernel.org:/pub/scm/linux/kernel/git/netdev/net") accidentally reverted the effect of commit 1a8024239da ("virtio-net: fix for skb_over_panic inside big mode") on drivers/net/virtio_net.c As a result, users of crosvm (which is using large packet mode) are experiencing crashes with 5.14-rc1 and above that do not occur with 5.13. Crash trace: [ 61.346677] skbuff: skb_over_panic: text:ffffffff881ae2c7 len:3762 put:3762 head:ffff8a5ec8c22000 data:ffff8a5ec8c22010 tail:0xec2 end:0xec0 dev:<NULL> [ 61.369192] kernel BUG at net/core/skbuff.c:111! [ 61.372840] invalid opcode: 0000 [#1] SMP PTI [ 61.374892] CPU: 5 PID: 0 Comm: swapper/5 Not tainted 5.14.0-rc1 linux-v5.14-rc1-for-mesa-ci.tar.bz2 #1 [ 61.376450] Hardware name: ChromiumOS crosvm, BIOS 0 Signed-off-by: Rob Bradford <[email protected]>
sboeuf
pushed a commit
that referenced
this pull request
Jan 6, 2022
The lock on the config should not be held whilst calling into
CpuManager::new().
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=24176)
Cycle in lock order graph: M13 (0x7b0c000001e0) => M43 (0x7b0c00001a70) => M13
Mutex M43 acquired here while holding mutex M13 in thread T1:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x663984)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x491bde)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::h8a843a6e74b34c4a /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36cd8b)
#4 vmm::cpu::CpuManager::new::h1cc88224a2a50d87 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:575:30 (cloud-hypervisor+0x372e16)
#5 vmm::vm::Vm::new_from_memory_manager::ha2a4467be260e93c /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:592:27 (cloud-hypervisor+0x330b86)
#6 vmm::vm::Vm::new::h43efe7c6cd97ede5 /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:784:22 (cloud-hypervisor+0x3343df)
#7 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:379:26 (cloud-hypervisor+0x2e5ba8)
#8 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f44e0)
#9 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1dda20)
#10 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x659bfa)
#11 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x44103e)
#12 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dda8e)
#13 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d3701)
#14 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d4748)
#15 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d2489)
#16 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57bdfe)
#17 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f943)
#18 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cf425)
#19 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d4c2)
#20 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d4c2)
#21 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d4c2)
Mutex M13 previously acquired by the same thread here:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x663984)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x491bde)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::ha29f58bbf496356a /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36d03b)
#4 vmm::vm::Vm::new_from_memory_manager::ha2a4467be260e93c /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:593:14 (cloud-hypervisor+0x3307ee)
#5 vmm::vm::Vm::new::h43efe7c6cd97ede5 /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:784:22 (cloud-hypervisor+0x3343df)
#6 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:379:26 (cloud-hypervisor+0x2e5ba8)
#7 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f44e0)
#8 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1dda20)
#9 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x659bfa)
#10 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x44103e)
#11 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dda8e)
#12 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d3701)
#13 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d4748)
#14 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d2489)
#15 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57bdfe)
#16 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f943)
#17 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cf425)
#18 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d4c2)
#19 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d4c2)
#20 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d4c2)
Mutex M13 acquired here while holding mutex M43 in thread T1:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x663984)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x491bde)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::ha29f58bbf496356a /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36d03b)
#4 vmm::device_manager::DeviceManager::add_console_device::h1d2b419feef80564 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:1839:29 (cloud-hypervisor+0x3972f4)
#5 vmm::device_manager::DeviceManager::create_devices::h29fc5b8a20e1aea5 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:1143:24 (cloud-hypervisor+0x38f068)
#6 vmm::vm::Vm::new::h43efe7c6cd97ede5 /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:798:9 (cloud-hypervisor+0x334671)
#7 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:379:26 (cloud-hypervisor+0x2e5ba8)
#8 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f44e0)
#9 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1dda20)
#10 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x659bfa)
#11 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x44103e)
#12 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dda8e)
#13 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d3701)
#14 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d4748)
#15 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d2489)
#16 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57bdfe)
#17 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f943)
#18 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cf425)
#19 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d4c2)
#20 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d4c2)
#21 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d4c2)
Mutex M43 previously acquired by the same thread here:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x663984)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x491bde)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::h8a843a6e74b34c4a /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36cd8b)
#4 vmm::vm::Vm::new::h43efe7c6cd97ede5 /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:798:9 (cloud-hypervisor+0x334537)
#5 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:379:26 (cloud-hypervisor+0x2e5ba8)
#6 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f44e0)
#7 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1dda20)
#8 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x659bfa)
#9 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x44103e)
#10 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dda8e)
#11 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d3701)
#12 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d4748)
#13 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d2489)
#14 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57bdfe)
#15 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f943)
#16 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cf425)
#17 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d4c2)
#18 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d4c2)
#19 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d4c2)
Signed-off-by: Rob Bradford <[email protected]>
sboeuf
pushed a commit
that referenced
this pull request
Jan 6, 2022
The allocator is locked by both the BAR allocation code and the
interrupt allocation code. Resulting in a potential lock inversion
error.
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=26318)
Cycle in lock order graph: M87 (0x7b0c00001e30) => M28 (0x7b0c00001830) => M87
Mutex M28 acquired here while holding mutex M87 in thread T1:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x663954)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x491bae)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::hc61622e5536f5b72 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36d07b)
#4 _$LT$vmm..interrupt..MsiInterruptManager$LT$kvm_bindings..x86..bindings..kvm_irq_routing_entry$GT$$u20$as$u20$vm_device..interrupt..InterruptManager$GT$::create_group::hd412b5e1e8eeacc2 /home/rob/src/cloud-hypervisor/vmm/src/interrupt.rs:310:29 (cloud-hypervisor+0x6d1403)
#5 virtio_devices::transport::pci_device::VirtioPciDevice::new::h3af603c3f00f4b3d /home/rob/src/cloud-hypervisor/virtio-devices/src/transport/pci_device.rs:376:38 (cloud-hypervisor+0x8e6137)
#6 vmm::device_manager::DeviceManager::add_virtio_pci_device::h23608151d7668a1c /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:3333:37 (cloud-hypervisor+0x3b6339)
#7 vmm::device_manager::DeviceManager::add_pci_devices::h136cc20cbeb6b977 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:1236:30 (cloud-hypervisor+0x390aad)
#8 vmm::device_manager::DeviceManager::create_devices::h29fc5b8a20e1aea5 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:1155:9 (cloud-hypervisor+0x38f48c)
#9 vmm::vm::Vm::new::h43efe7c6cd97ede5 /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:799:9 (cloud-hypervisor+0x334641)
#10 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:379:26 (cloud-hypervisor+0x2e5ba8)
#11 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f44e0)
#12 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1dda20)
#13 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x659bca)
#14 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x44100e)
#15 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dda5e)
#16 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d36d1)
#17 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d4718)
#18 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d2459)
#19 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57bdce)
#20 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f913)
#21 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cf3f5)
#22 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d492)
#23 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d492)
#24 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d492)
Mutex M87 previously acquired by the same thread here:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::h9a2d3e97e05c6430 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x9ea344)
#2 std::sys_common::mutex::MovableMutex::raw_lock::h8abb3b5cf55c0264 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x96face)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::hecec128d40c6dd44 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x97120a)
#4 virtio_devices::transport::pci_device::VirtioPciDevice::new::h3af603c3f00f4b3d /home/rob/src/cloud-hypervisor/virtio-devices/src/transport/pci_device.rs:356:29 (cloud-hypervisor+0x8e5c0e)
#5 vmm::device_manager::DeviceManager::add_virtio_pci_device::h23608151d7668a1c /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:3333:37 (cloud-hypervisor+0x3b6339)
#6 vmm::device_manager::DeviceManager::add_pci_devices::h136cc20cbeb6b977 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:1236:30 (cloud-hypervisor+0x390aad)
#7 vmm::device_manager::DeviceManager::create_devices::h29fc5b8a20e1aea5 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:1155:9 (cloud-hypervisor+0x38f48c)
#8 vmm::vm::Vm::new::h43efe7c6cd97ede5 /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:799:9 (cloud-hypervisor+0x334641)
#9 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:379:26 (cloud-hypervisor+0x2e5ba8)
#10 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f44e0)
#11 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1dda20)
#12 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x659bca)
#13 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x44100e)
#14 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dda5e)
#15 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d36d1)
#16 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d4718)
#17 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d2459)
#18 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57bdce)
#19 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f913)
#20 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cf3f5)
#21 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d492)
#22 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d492)
#23 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d492)
Mutex M87 acquired here while holding mutex M28 in thread T1:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::h9a2d3e97e05c6430 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x9ea344)
#2 std::sys_common::mutex::MovableMutex::raw_lock::h8abb3b5cf55c0264 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x96face)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::hecec128d40c6dd44 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x97120a)
#4 _$LT$virtio_devices..transport..pci_device..VirtioPciDevice$u20$as$u20$pci..device..PciDevice$GT$::allocate_bars::h39dc42b48fc8264c /home/rob/src/cloud-hypervisor/virtio-devices/src/transport/pci_device.rs:850:22 (cloud-hypervisor+0x8eb1a4)
#5 vmm::device_manager::DeviceManager::add_pci_device::h561f6c8ed61db117 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:3087:20 (cloud-hypervisor+0x3b0c62)
#6 vmm::device_manager::DeviceManager::add_virtio_pci_device::h23608151d7668a1c /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:3359:20 (cloud-hypervisor+0x3b6707)
#7 vmm::device_manager::DeviceManager::add_pci_devices::h136cc20cbeb6b977 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:1236:30 (cloud-hypervisor+0x390aad)
#8 vmm::device_manager::DeviceManager::create_devices::h29fc5b8a20e1aea5 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:1155:9 (cloud-hypervisor+0x38f48c)
#9 vmm::vm::Vm::new::h43efe7c6cd97ede5 /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:799:9 (cloud-hypervisor+0x334641)
#10 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:379:26 (cloud-hypervisor+0x2e5ba8)
#11 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f44e0)
#12 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1dda20)
#13 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x659bca)
#14 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x44100e)
#15 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dda5e)
#16 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d36d1)
#17 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d4718)
#18 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d2459)
#19 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57bdce)
#20 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f913)
#21 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cf3f5)
#22 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d492)
#23 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d492)
#24 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d492)
Mutex M28 previously acquired by the same thread here:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x663954)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x491bae)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::hc61622e5536f5b72 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36d07b)
#4 vmm::device_manager::DeviceManager::add_pci_device::h561f6c8ed61db117 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:3091:22 (cloud-hypervisor+0x3b0a95)
#5 vmm::device_manager::DeviceManager::add_virtio_pci_device::h23608151d7668a1c /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:3359:20 (cloud-hypervisor+0x3b6707)
#6 vmm::device_manager::DeviceManager::add_pci_devices::h136cc20cbeb6b977 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:1236:30 (cloud-hypervisor+0x390aad)
#7 vmm::device_manager::DeviceManager::create_devices::h29fc5b8a20e1aea5 /home/rob/src/cloud-hypervisor/vmm/src/device_manager.rs:1155:9 (cloud-hypervisor+0x38f48c)
#8 vmm::vm::Vm::new::h43efe7c6cd97ede5 /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:799:9 (cloud-hypervisor+0x334641)
#9 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:379:26 (cloud-hypervisor+0x2e5ba8)
#10 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f44e0)
#11 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1dda20)
#12 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x659bca)
#13 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x44100e)
#14 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dda5e)
#15 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d36d1)
#16 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d4718)
#17 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d2459)
#18 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57bdce)
#19 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f913)
#20 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cf3f5)
#21 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d492)
#22 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d492)
#23 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d492)
Signed-off-by: Rob Bradford <[email protected]>
sboeuf
pushed a commit
that referenced
this pull request
Jan 6, 2022
Delay creating the mutex on the Vcpu until later preventing a potential
lock inversion between CpuManager and the Vcpu.
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=28799)
Cycle in lock order graph: M48 (0x7b0c00001aa0) => M117 (0x7b0c00002280) => M48
Mutex M117 acquired here while holding mutex M48 in thread T1:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x662fc4)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x4915ae)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::h14cfa3c8f5ba878a /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36c82b)
#4 vmm::cpu::CpuManager::create_vcpu::hd5878da6efae8d68 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:665:13 (cloud-hypervisor+0x3743de)
#5 vmm::cpu::CpuManager::create_vcpus::h3c747553a1d5bc4e /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:704:13 (cloud-hypervisor+0x374d87)
#6 vmm::cpu::CpuManager::create_boot_vcpus::he8eeca10785067c1 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:938:9 (cloud-hypervisor+0x377305)
#7 vmm::vm::Vm::boot::hc2ca6b16f996267b /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:1986:9 (cloud-hypervisor+0x3432d3)
#8 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:397:13 (cloud-hypervisor+0x2e5d35)
#9 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f42d0)
#10 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1dd8d0)
#11 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x65923a)
#12 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x440a0e)
#13 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dd0ce)
#14 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2d41)
#15 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3d88)
#16 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d1ac9)
#17 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57b72e)
#18 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f313)
#19 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cef15)
#20 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d662)
#21 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d662)
#22 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d662)
Mutex M48 previously acquired by the same thread here:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x662fc4)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x4915ae)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::hecf671add5fe1762 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36cfbb)
#4 vmm::vm::Vm::boot::hc2ca6b16f996267b /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:1986:9 (cloud-hypervisor+0x343237)
#5 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:397:13 (cloud-hypervisor+0x2e5d35)
#6 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f42d0)
#7 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1dd8d0)
#8 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x65923a)
#9 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x440a0e)
#10 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dd0ce)
#11 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2d41)
#12 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3d88)
#13 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d1ac9)
#14 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57b72e)
#15 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f313)
#16 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cef15)
#17 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d662)
#18 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d662)
#19 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d662)
Mutex M48 acquired here while holding mutex M117 in thread T3:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::h967991d72ceb6eb0 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0xd94dc4)
#2 std::sys_common::mutex::MovableMutex::raw_lock::h8779639163126a21 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0xd90c9e)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::hd85239d207beb12f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0xd6e98a)
#4 vm_device::bus::Bus::write::hf20f991e71af3199 /home/rob/src/cloud-hypervisor/vm-device/src/bus.rs:235:16 (cloud-hypervisor+0xd8dcfd)
#5 _$LT$vmm..vm..VmOps$u20$as$u20$hypervisor..vm..VmmOps$GT$::mmio_write::hc759194aaebc7399 /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:424:15 (cloud-hypervisor+0x32d94f)
#6 _$LT$hypervisor..kvm..KvmVcpu$u20$as$u20$hypervisor..cpu..Vcpu$GT$::run::h94762dfba6642fb2 /home/rob/src/cloud-hypervisor/hypervisor/src/kvm/mod.rs:1003:32 (cloud-hypervisor+0xcc3ea8)
#7 vmm::cpu::Vcpu::run::hd5cf042157f95bea /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:327:9 (cloud-hypervisor+0x3700c4)
#8 vmm::cpu::CpuManager::start_vcpu::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h37e4dd8619b3a5e5 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:819:35 (cloud-hypervisor+0x47782b)
#9 std::panicking::try::do_call::h093e4d1434150d77 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2aba)
#10 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3d88)
#11 std::panicking::try::hee9535cb997282b4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d265f)
#12 std::panic::catch_unwind::he3908c4d08a8a028 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57bac9)
#13 vmm::cpu::CpuManager::start_vcpu::_$u7b$$u7b$closure$u7d$$u7d$::h29472aaa3a600231 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:788:21 (cloud-hypervisor+0x477126)
#14 std::sys_common::backtrace::__rust_begin_short_backtrace::hcfc2f02361c98808 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x6592fb)
#15 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h08b82db41d7af2f2 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x4408bf)
#16 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h7ebad9d94e64fa5f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dd1af)
#17 std::panicking::try::do_call::h121fafbdf5cf84af /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2b51)
#18 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3d88)
#19 std::panicking::try::h79e25f019cd90522 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d1def)
#20 std::panic::catch_unwind::h5a0619a53bbd611d /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57b8cf)
#21 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h1cfd689c9d362e48 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43facb)
#22 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h6642b1b3a2289640 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3ced25)
#23 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d662)
#24 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d662)
#25 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d662)
Mutex M117 previously acquired by the same thread here:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x662fc4)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x4915ae)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::h14cfa3c8f5ba878a /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36c82b)
#4 vmm::cpu::CpuManager::start_vcpu::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h37e4dd8619b3a5e5 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:819:35 (cloud-hypervisor+0x477799)
#5 std::panicking::try::do_call::h093e4d1434150d77 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2aba)
#6 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3d88)
#7 std::panicking::try::hee9535cb997282b4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d265f)
#8 std::panic::catch_unwind::he3908c4d08a8a028 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57bac9)
#9 vmm::cpu::CpuManager::start_vcpu::_$u7b$$u7b$closure$u7d$$u7d$::h29472aaa3a600231 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:788:21 (cloud-hypervisor+0x477126)
#10 std::sys_common::backtrace::__rust_begin_short_backtrace::hcfc2f02361c98808 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x6592fb)
#11 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h08b82db41d7af2f2 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x4408bf)
#12 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h7ebad9d94e64fa5f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dd1af)
#13 std::panicking::try::do_call::h121fafbdf5cf84af /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2b51)
#14 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3d88)
#15 std::panicking::try::h79e25f019cd90522 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d1def)
#16 std::panic::catch_unwind::h5a0619a53bbd611d /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57b8cf)
#17 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h1cfd689c9d362e48 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43facb)
#18 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h6642b1b3a2289640 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3ced25)
#19 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d662)
#20 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d662)
#21 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d662)
Signed-off-by: Rob Bradford <[email protected]>
sboeuf
pushed a commit
that referenced
this pull request
Jan 6, 2022
Remove requirement for CpuManager to lock the Vcpu when starting the
vCPU as the numerical id corresponds to the index in the the vector.
This avoids a potential lock inversion between the Vcpu and CpuManager.
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=30497)
Cycle in lock order graph: M48 (0x7b0c00001aa0) => M121 (0x7b0c000022b0) => M48
Mutex M121 acquired here while holding mutex M48 in thread T1:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x662ff4)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x4915de)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::h14cfa3c8f5ba878a /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36ca3b)
#4 vmm::cpu::CpuManager::start_vcpu::h290fdbb4b7124ec5 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:710:22 (cloud-hypervisor+0x375023)
#5 vmm::cpu::CpuManager::activate_vcpus::h2eab380826588391 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:902:13 (cloud-hypervisor+0x376b7b)
#6 vmm::cpu::CpuManager::start_boot_vcpus::hd80cafe6aa4e8279 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:937:9 (cloud-hypervisor+0x3773af)
#7 vmm::vm::Vm::boot::hc2ca6b16f996267b /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:2063:9 (cloud-hypervisor+0x343d57)
#8 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:397:13 (cloud-hypervisor+0x2e5f45)
#9 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f44e0)
#10 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1ddae0)
#11 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x65926a)
#12 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x440a3e)
#13 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dd0fe)
#14 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2d71)
#15 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3db8)
#16 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d1af9)
#17 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57b75e)
#18 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f343)
#19 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cef45)
#20 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d692)
#21 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d692)
#22 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d692)
Mutex M48 previously acquired by the same thread here:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x662ff4)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x4915de)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::hecf671add5fe1762 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36d1cb)
#4 vmm::vm::Vm::boot::hc2ca6b16f996267b /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:2063:9 (cloud-hypervisor+0x343cd1)
#5 vmm::Vmm::vm_boot::h06bdf54b95d5e14f /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:397:13 (cloud-hypervisor+0x2e5f45)
#6 vmm::Vmm::control_loop::h40c9b48c7b800bed /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:1299:48 (cloud-hypervisor+0x2f44e0)
#7 vmm::start_vmm_thread::_$u7b$$u7b$closure$u7d$$u7d$::h016d2f7cff698175 /home/rob/src/cloud-hypervisor/vmm/src/lib.rs:263:17 (cloud-hypervisor+0x1ddae0)
#8 std::sys_common::backtrace::__rust_begin_short_backtrace::h7fd2df3e7cfba503 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x65926a)
#9 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h89880b05fe892d7e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x440a3e)
#10 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h487382524d80571f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dd0fe)
#11 std::panicking::try::do_call::h1d9c2ccdc39f3322 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2d71)
#12 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3db8)
#13 std::panicking::try::h251306df23d21913 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d1af9)
#14 std::panic::catch_unwind::h2a9ac2fb12c3c64e /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57b75e)
#15 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h10f4c340611b55e4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43f343)
#16 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hdd9b37241caf97b3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3cef45)
#17 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d692)
#18 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d692)
#19 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d692)
Mutex M48 acquired here while holding mutex M121 in thread T4:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::h967991d72ceb6eb0 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0xd94df4)
#2 std::sys_common::mutex::MovableMutex::raw_lock::h8779639163126a21 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0xd90cce)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::hd85239d207beb12f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0xd6e9ba)
#4 vm_device::bus::Bus::write::hf20f991e71af3199 /home/rob/src/cloud-hypervisor/vm-device/src/bus.rs:235:16 (cloud-hypervisor+0xd8dd2d)
#5 _$LT$vmm..vm..VmOps$u20$as$u20$hypervisor..vm..VmmOps$GT$::mmio_write::hc759194aaebc7399 /home/rob/src/cloud-hypervisor/vmm/src/vm.rs:424:15 (cloud-hypervisor+0x32db5f)
#6 _$LT$hypervisor..kvm..KvmVcpu$u20$as$u20$hypervisor..cpu..Vcpu$GT$::run::h94762dfba6642fb2 /home/rob/src/cloud-hypervisor/hypervisor/src/kvm/mod.rs:1003:32 (cloud-hypervisor+0xcc3ed8)
#7 vmm::cpu::Vcpu::run::hd5cf042157f95bea /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:327:9 (cloud-hypervisor+0x370234)
#8 vmm::cpu::CpuManager::start_vcpu::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h37e4dd8619b3a5e5 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:813:35 (cloud-hypervisor+0x47785b)
#9 std::panicking::try::do_call::h093e4d1434150d77 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2aea)
#10 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3db8)
#11 std::panicking::try::hee9535cb997282b4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d268f)
#12 std::panic::catch_unwind::he3908c4d08a8a028 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57baf9)
#13 vmm::cpu::CpuManager::start_vcpu::_$u7b$$u7b$closure$u7d$$u7d$::h29472aaa3a600231 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:782:21 (cloud-hypervisor+0x477156)
#14 std::sys_common::backtrace::__rust_begin_short_backtrace::hcfc2f02361c98808 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x65932b)
#15 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h08b82db41d7af2f2 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x4408ef)
#16 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h7ebad9d94e64fa5f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dd1df)
#17 std::panicking::try::do_call::h121fafbdf5cf84af /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2b81)
#18 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3db8)
#19 std::panicking::try::h79e25f019cd90522 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d1e1f)
#20 std::panic::catch_unwind::h5a0619a53bbd611d /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57b8ff)
#21 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h1cfd689c9d362e48 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43fafb)
#22 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h6642b1b3a2289640 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3ced55)
#23 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d692)
#24 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d692)
#25 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d692)
Mutex M121 previously acquired by the same thread here:
#0 pthread_mutex_lock /rustc/llvm/src/llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4249:3 (cloud-hypervisor+0x9c368)
#1 std::sys::unix::mutex::Mutex::lock::hcd1b9aa06ff775d3 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/mutex.rs:63:17 (cloud-hypervisor+0x662ff4)
#2 std::sys_common::mutex::MovableMutex::raw_lock::hff98d0b036469bca /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/mutex.rs:76:18 (cloud-hypervisor+0x4915de)
#3 std::sync::mutex::Mutex$LT$T$GT$::lock::h14cfa3c8f5ba878a /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sync/mutex.rs:267:13 (cloud-hypervisor+0x36ca3b)
#4 vmm::cpu::CpuManager::start_vcpu::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h37e4dd8619b3a5e5 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:813:35 (cloud-hypervisor+0x4777c9)
#5 std::panicking::try::do_call::h093e4d1434150d77 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2aea)
#6 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3db8)
#7 std::panicking::try::hee9535cb997282b4 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d268f)
#8 std::panic::catch_unwind::he3908c4d08a8a028 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57baf9)
#9 vmm::cpu::CpuManager::start_vcpu::_$u7b$$u7b$closure$u7d$$u7d$::h29472aaa3a600231 /home/rob/src/cloud-hypervisor/vmm/src/cpu.rs:782:21 (cloud-hypervisor+0x477156)
#10 std::sys_common::backtrace::__rust_begin_short_backtrace::hcfc2f02361c98808 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys_common/backtrace.rs:123:18 (cloud-hypervisor+0x65932b)
#11 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h08b82db41d7af2f2 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:477:17 (cloud-hypervisor+0x4408ef)
#12 _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h7ebad9d94e64fa5f /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panic/unwind_safe.rs:271:9 (cloud-hypervisor+0x6dd1df)
#13 std::panicking::try::do_call::h121fafbdf5cf84af /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:406:40 (cloud-hypervisor+0x6d2b81)
#14 __rust_try 3hkmq3dzyyv5ejsx (cloud-hypervisor+0x6d3db8)
#15 std::panicking::try::h79e25f019cd90522 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:370:19 (cloud-hypervisor+0x6d1e1f)
#16 std::panic::catch_unwind::h5a0619a53bbd611d /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panic.rs:133:14 (cloud-hypervisor+0x57b8ff)
#17 std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h1cfd689c9d362e48 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/thread/mod.rs:476:30 (cloud-hypervisor+0x43fafb)
#18 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h6642b1b3a2289640 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5 (cloud-hypervisor+0x3ced55)
#19 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::ha5022a6bb7833f62 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d692)
#20 _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h481697829cbc6746 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/alloc/src/boxed.rs:1854:9 (cloud-hypervisor+0x119d692)
#21 std::sys::unix::thread::Thread::new::thread_start::h6fad62c4c393bbe7 /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/sys/unix/thread.rs:108:17 (cloud-hypervisor+0x119d692)
Signed-off-by: Rob Bradford <[email protected]>
up2wing
pushed a commit
to up2wing/cloud-hypervisor
that referenced
this pull request
Mar 9, 2022
When mask a msi irq, we set the entry.masked to be true, so kvm hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING ioctl which update kvm->irq_routing. This will trigger kernel panic on AMD platform when the gsi is the largest one in kernel kvm->irqfds.items: crash> bt PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8" #0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397 cloud-hypervisor#1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d cloud-hypervisor#2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d cloud-hypervisor#3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d cloud-hypervisor#4 [ffffb1ba6707fb90] no_context at ffffffff856692c9 cloud-hypervisor#5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51 cloud-hypervisor#6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace [exception RIP: svm_update_pi_irte+227] RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086 RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001 RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8 RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200 R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001 R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 cloud-hypervisor#7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm] cloud-hypervisor#8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm] cloud-hypervisor#9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm] RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246 RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020 RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0 R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0 R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0 ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b To solve this problem, move route.disable() before set_gsi_routes() to remove the gsi from irqfds.items first. This problem only exists on AMD platform, 'cause on Intel platform kernel just return when update irte while it only prints a warning on AMD. Signed-off-by: Yi Wang <[email protected]>
up2wing
pushed a commit
to up2wing/cloud-hypervisor
that referenced
this pull request
Mar 10, 2022
When mask a msi irq, we set the entry.masked to be true, so kvm hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING ioctl which update kvm->irq_routing. This will trigger kernel panic on AMD platform when the gsi is the largest one in kernel kvm->irqfds.items: crash> bt PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8" #0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397 cloud-hypervisor#1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d cloud-hypervisor#2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d cloud-hypervisor#3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d cloud-hypervisor#4 [ffffb1ba6707fb90] no_context at ffffffff856692c9 cloud-hypervisor#5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51 cloud-hypervisor#6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace [exception RIP: svm_update_pi_irte+227] RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086 RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001 RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8 RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200 R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001 R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 cloud-hypervisor#7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm] cloud-hypervisor#8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm] cloud-hypervisor#9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm] RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246 RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020 RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0 R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0 R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0 ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b To solve this problem, move route.disable() before set_gsi_routes() to remove the gsi from irqfds.items first. This problem only exists on AMD platform, 'cause on Intel platform kernel just return when update irte while it only prints a warning on AMD. Also, this patch adjusts the order of enable() and set_gsi_routes() in unmask(), which should do no harm. Signed-off-by: Yi Wang <[email protected]>
sboeuf
pushed a commit
that referenced
this pull request
Mar 10, 2022
When mask a msi irq, we set the entry.masked to be true, so kvm hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING ioctl which update kvm->irq_routing. This will trigger kernel panic on AMD platform when the gsi is the largest one in kernel kvm->irqfds.items: crash> bt PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8" #0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397 #1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d #2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d #3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d #4 [ffffb1ba6707fb90] no_context at ffffffff856692c9 #5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51 #6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace [exception RIP: svm_update_pi_irte+227] RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086 RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001 RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8 RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200 R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001 R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm] #8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm] #9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm] RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246 RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020 RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0 R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0 R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0 ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b To solve this problem, move route.disable() before set_gsi_routes() to remove the gsi from irqfds.items first. This problem only exists on AMD platform, 'cause on Intel platform kernel just return when update irte while it only prints a warning on AMD. Signed-off-by: Yi Wang <[email protected]>
sboeuf
pushed a commit
that referenced
this pull request
Mar 10, 2022
When mask a msi irq, we set the entry.masked to be true, so kvm hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING ioctl which update kvm->irq_routing. This will trigger kernel panic on AMD platform when the gsi is the largest one in kernel kvm->irqfds.items: crash> bt PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8" #0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397 #1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d #2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d #3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d #4 [ffffb1ba6707fb90] no_context at ffffffff856692c9 #5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51 #6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace [exception RIP: svm_update_pi_irte+227] RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086 RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001 RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8 RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200 R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001 R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm] #8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm] #9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm] RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246 RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020 RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0 R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0 R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0 ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b To solve this problem, move route.disable() before set_gsi_routes() to remove the gsi from irqfds.items first. This problem only exists on AMD platform, 'cause on Intel platform kernel just return when update irte while it only prints a warning on AMD. Also, this patch adjusts the order of enable() and set_gsi_routes() in unmask(), which should do no harm. Signed-off-by: Yi Wang <[email protected]>
rbradford
pushed a commit
that referenced
this pull request
Mar 11, 2022
When mask a msi irq, we set the entry.masked to be true, so kvm hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING ioctl which update kvm->irq_routing. This will trigger kernel panic on AMD platform when the gsi is the largest one in kernel kvm->irqfds.items: crash> bt PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8" #0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397 #1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d #2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d #3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d #4 [ffffb1ba6707fb90] no_context at ffffffff856692c9 #5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51 #6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace [exception RIP: svm_update_pi_irte+227] RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086 RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001 RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8 RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200 R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001 R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm] #8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm] #9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm] RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246 RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020 RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0 R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0 R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0 ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b To solve this problem, move route.disable() before set_gsi_routes() to remove the gsi from irqfds.items first. This problem only exists on AMD platform, 'cause on Intel platform kernel just return when update irte while it only prints a warning on AMD. Signed-off-by: Yi Wang <[email protected]> (cherry picked from commit db9e5e5) Signed-off-by: Rob Bradford <[email protected]>
rbradford
pushed a commit
that referenced
this pull request
Mar 11, 2022
When mask a msi irq, we set the entry.masked to be true, so kvm hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING ioctl which update kvm->irq_routing. This will trigger kernel panic on AMD platform when the gsi is the largest one in kernel kvm->irqfds.items: crash> bt PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8" #0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397 #1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d #2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d #3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d #4 [ffffb1ba6707fb90] no_context at ffffffff856692c9 #5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51 #6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace [exception RIP: svm_update_pi_irte+227] RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086 RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001 RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8 RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200 R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001 R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm] #8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm] #9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm] RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246 RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020 RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0 R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0 R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0 ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b To solve this problem, move route.disable() before set_gsi_routes() to remove the gsi from irqfds.items first. This problem only exists on AMD platform, 'cause on Intel platform kernel just return when update irte while it only prints a warning on AMD. Also, this patch adjusts the order of enable() and set_gsi_routes() in unmask(), which should do no harm. Signed-off-by: Yi Wang <[email protected]> (cherry picked from commit 5375b84) Signed-off-by: Rob Bradford <[email protected]>
rbradford
pushed a commit
that referenced
this pull request
Mar 11, 2022
When mask a msi irq, we set the entry.masked to be true, so kvm hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING ioctl which update kvm->irq_routing. This will trigger kernel panic on AMD platform when the gsi is the largest one in kernel kvm->irqfds.items: crash> bt PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8" #0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397 #1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d #2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d #3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d #4 [ffffb1ba6707fb90] no_context at ffffffff856692c9 #5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51 #6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace [exception RIP: svm_update_pi_irte+227] RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086 RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001 RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8 RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200 R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001 R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm] #8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm] #9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm] RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246 RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020 RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0 R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0 R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0 ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b To solve this problem, move route.disable() before set_gsi_routes() to remove the gsi from irqfds.items first. This problem only exists on AMD platform, 'cause on Intel platform kernel just return when update irte while it only prints a warning on AMD. Signed-off-by: Yi Wang <[email protected]> (cherry picked from commit db9e5e5) Signed-off-by: Rob Bradford <[email protected]>
rbradford
pushed a commit
that referenced
this pull request
Mar 11, 2022
When mask a msi irq, we set the entry.masked to be true, so kvm hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING ioctl which update kvm->irq_routing. This will trigger kernel panic on AMD platform when the gsi is the largest one in kernel kvm->irqfds.items: crash> bt PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8" #0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397 #1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d #2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d #3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d #4 [ffffb1ba6707fb90] no_context at ffffffff856692c9 #5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51 #6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace [exception RIP: svm_update_pi_irte+227] RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086 RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001 RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8 RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200 R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001 R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm] #8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm] #9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm] RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246 RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020 RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0 R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0 R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0 ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b To solve this problem, move route.disable() before set_gsi_routes() to remove the gsi from irqfds.items first. This problem only exists on AMD platform, 'cause on Intel platform kernel just return when update irte while it only prints a warning on AMD. Also, this patch adjusts the order of enable() and set_gsi_routes() in unmask(), which should do no harm. Signed-off-by: Yi Wang <[email protected]> (cherry picked from commit 5375b84) Signed-off-by: Rob Bradford <[email protected]>
This was referenced Nov 2, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Samuel Ortiz [email protected]