Reprogram PCI device BAR when its MSE bit is set #7063
Conversation
likebreath
force-pushed
the
0512/fix_move_bar
branch
from
aeadb36 to
5621106
Compare
rbradford
left a comment
rbradford
left a comment
There was a problem hiding this comment.
Looks good - did you consider storing the pending program per bar rather than all the BARs together per device? Would that actually end up being more complicated?
I think it should be pretty straight forward, while the code will be a bit more complicated (say using a Two potential benefits I can see:
Also both benefits appear to be minor. Anything I am missing? |
Rather than an external store like a hash table or vector I was thinking that the pending value could be stored on the |
The Memory Space Enable (MSE) bit from the COMMAND register in the PCI configuration space controls whether a PCI device responds to memory space accesses, e.g. read and write cycles to the device MMIO regions defined by its BARs. The MSE bit is used by the device drivers to ensure the correctness of BAR reprogramming. A common workflow is, the driver first clears the MSE bit, then writes new values to the BAR registers, and finally set the MSE bit to finish the BAR reprogramming. This patch changes how we handle BAR reprogramming for all PCI devices (e.g. virtio-pci, vfio, vfio-user, etc.), so that we follow the same convention, e.g. moving PCI BARs when its MSE bit is set. Note that some device drivers (such as edk2) only clear and set MSE once while reprogramming multiple BARs of a single device. To support such behavior, this patch adds support for multiple pending BAR reprogramming. See: cloud-hypervisor#7027 (comment) Signed-off-by: Bo Chen <[email protected]>
The way how we handle PCI configuration space for vfio and vfio-user devices are different from the rest of PCI devices. Besides accesses to BAR registers (trapped to access the shadowing PCI config space we maintained), accesses to other registers (including the COMMAND register) are handled directly by the underline vfio or vfio-user device. This patch adds the proper handling of pending BAR reprogramming for vfio and vfio-user devices. Signed-off-by: Bo Chen <[email protected]>
likebreath
force-pushed
the
0512/fix_move_bar
branch
from
5621106 to
d88a8e0
Compare
|
We discussed about this PR in the community meeting today. We agreed it is the right way to go, considering more refactoring needed and potential impacts to live-migration with alternative approaches. |
8da7c13
2 participants
Note: this PR depends on #7057.
The Memory Space Enable (MSE) bit from the COMMAND register in the
PCI configuration space controls whether a PCI device responds to memory
space accesses, e.g. read and write cycles to the device MMIO regions
defined by its BARs. The MSE bit is used by the device drivers to ensure
the correctness of BAR reprogramming. A common workflow is, the driver
first clears the MSE bit, then writes new values to the BAR registers,
and finally set the MSE bit to finish the BAR reprogramming.
This patch changes how we handle BAR reprogramming for all PCI
devices (e.g. virtio-pci, vfio, vfio-user, etc.), so that we follow the
same convention, e.g. moving PCI BARs when its MSE bit is set.
Note that some device drivers (such as edk2) only clear and set MSE once
while reprogramming multiple BARs of a single device. To support such
behavior, this patch adds support for multiple pending BAR reprogramming.
See: #7027 (comment)
Also, the way how we handle PCI configuration space for vfio and vfio-user
devices are different from the rest of PCI devices. Besides accesses to
BAR registers (trapped to access the shadowing PCI config space we
maintained), accesses to other registers (including the COMMAND
register) are handled directly by the underline vfio or vfio-user
device.
This patch adds the proper handling of pending BAR reprogramming for
vfio and vfio-user devices.