OASIS Virtual I/O Device (VIRTIO) TC

  • 1.  virtio-pci physical addresses vs bus addresses

    Posted 09-12-2013 12:27
    The vring has traditionally used physical addresses since the hypervisor has access to guest RAM. As virtio continues to be deployed in new ways, including as real hardware, I wonder if physical addresses will become a hindrance. Nested virtualization also poses a problem because virtio devices cannot be passed through to L2 or deeper guests. The L2 guest has no concept of the physical address that the L0 hypervisor expects. Given the other PCI changes that are being proposed for the virtio standard, this might be the right opportunity to make virtio-pci behave like a real PCI adapter with respect to DMA and bus addresses. Of course this may add overhead to existing virtio implementations since emulating real DMA on modern systems can mean an extra layer of indirection (IO-MMU). At the same time, we need to be aware of the limitations of using physical addresses directly. Thoughts? Stefan


  • 2.  Re: [virtio] virtio-pci physical addresses vs bus addresses

    Posted 09-12-2013 12:55
    On Thu, Sep 12, 2013 at 02:27:20PM +0200, Stefan Hajnoczi wrote: > The vring has traditionally used physical addresses since the hypervisor > has access to guest RAM. > > As virtio continues to be deployed in new ways, including as real > hardware, I wonder if physical addresses will become a hindrance. > > Nested virtualization also poses a problem because virtio devices cannot > be passed through to L2 or deeper guests. The L2 guest has no concept > of the physical address that the L0 hypervisor expects. > > Given the other PCI changes that are being proposed for the virtio > standard, this might be the right opportunity to make virtio-pci behave > like a real PCI adapter with respect to DMA and bus addresses. > > Of course this may add overhead to existing virtio implementations since > emulating real DMA on modern systems can mean an extra layer of > indirection (IO-MMU). > > At the same time, we need to be aware of the limitations of using > physical addresses directly. > > Thoughts? > > Stefan I'm inclined to leave it with physical address by default. I don't really see a pressing need to pass-through a virtio device, even in a nested virt setting. On the other hand, in virt setting people are enabling IOMMU, and it applies to all devices by default. The overhead of going through that is significant. We can always add this optionally as a feature bit later if there's interest. -- MST


  • 3.  Re: [virtio] virtio-pci physical addresses vs bus addresses

    Posted 09-12-2013 12:57
    On Thu, Sep 12, 2013 at 02:27:20PM +0200, Stefan Hajnoczi wrote:
    > The vring has traditionally used physical addresses since the hypervisor
    > has access to guest RAM.
    >
    > As virtio continues to be deployed in new ways, including as real
    > hardware, I wonder if physical addresses will become a hindrance.
    >
    > Nested virtualization also poses a problem because virtio devices cannot
    > be passed through to L2 or deeper guests. The L2 guest has no concept
    > of the physical address that the L0 hypervisor expects.
    >
    > Given the other PCI changes that are being proposed for the virtio
    > standard, this might be the right opportunity to make virtio-pci behave
    > like a real PCI adapter with respect to DMA and bus addresses.
    >
    > Of course this may add overhead to existing virtio implementations since
    > emulating real DMA on modern systems can mean an extra layer of
    > indirection (IO-MMU).
    >
    > At the same time, we need to be aware of the limitations of using
    > physical addresses directly.
    >
    > Thoughts?
    >
    > Stefan

    I'm inclined to leave it with physical address by default. I don't
    really see a pressing need to pass-through a virtio device, even in a
    nested virt setting.
    On the other hand, in virt setting people are enabling IOMMU,
    and it applies to all devices by default.
    The overhead of going through that is significant.

    We can always add this optionally as a feature bit later
    if there's interest.

    --
    MST