virtio-comment

 View Only

Re: PCI cap for larger offsets/lengths

  • 1.  Re: PCI cap for larger offsets/lengths

    Posted 11-26-2018 14:04
    * Gerd Hoffmann (kraxel@redhat.com) wrote:
    > On Mon, Nov 26, 2018 at 11:16:12AM +0000, Stefan Hajnoczi wrote:
    > > On Fri, Sep 21, 2018 at 10:54:59AM +0100, Dr. David Alan Gilbert wrote:
    > > > Hi,
    > > > We've got an experimental virtio device (using vhost-user) we're playing with
    > > > that would like to share multiple large mappings from the client back to qemu.
    > >
    > > CCing Michael Tsirkin and Gerd Hoffman. Gerd could use this for
    > > virtio-gpu where some memory must be owned by the host.
    >
    > Yep. For virtio-gpu I want be able to map host gpu resources (which
    > must be allocated by the host gpu driver) into the guest address space.
    >
    > > > 'virtio_pci_cap' only has 32bit offset and length fields and so
    > > > I've got a different capability to express larger regions:
    > > >
    > > >
    > > > /* Additional shared memory capability */
    > > > #define VIRTIO_PCI_CAP_SHARED_MEMORY_CFG 8
    > > >
    > > > struct virtio_pci_shm_cap {
    > > > struct virtio_pci_cap cap;
    > > > le32 offset_hi; /* Most sig 32 bits of offset */
    > > > le32 length_hi; /* Most sig 32 bits of length */
    > > > u8 id; /* To distinguish shm chunks */
    > > > };
    > > >
    > > > One oddity is that I'm allowing multiple instances of this capability
    > > > on one device, distinguished by their 'id' field which I've made device
    > > > type specific, e.g.:
    > > >
    > > > #define VIRTIO_MYDEV_PCI_SHMCAP_ID_CACHE 0
    > > > #define VIRTIO_MYDEV_PCI_SHMCAP_ID_JOURNAL 1
    >
    > For my experimental virtio-gpu code I use one pci bar to reserve address
    > space. It is a separate pci bar. First, because it is a 64bit bar.
    > Second, because it is declared as prefetchable (unlike the mmio bar
    > which is not). I also simply use the whole bar, so no offset/length is
    > needed.
    >
    > gpu resources are sub-regions within that pci bar, and they are managed
    > using device-specific commands.
    >
    > So, I'm wondering whenever it makes sense to just do the same for your
    > device. Just use one pci bar as shared memory umbrella, specify that
    > one using the virtio vendor cap, then have sub-regions within that bar
    > for the various regions you have. Manage them dynamically (using
    > device-specific virtio commands) or just have a static configuration (in
    > device-specific config space).

    Ours are static subdivisions; so it felt easier to declare them; it's a
    shame to make that device specific.

    > That avoids the problem with multiple capabilities of the same kind, and
    > it also avoids exhausting the cap IDs quicky if every device defines
    > their own VIRTIO_FOO_DEVICE_PCI_SHMCAP_ID_BAR_REGION.

    Is having multiple capabilities of the same type actually a problem, or
    is it just historical in the defitinition of virtio?

    Dave

    > cheers,
    > Gerd
    >
    --
    Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK