OASIS Virtual I/O Device (VIRTIO) TC

 View Only
  • 1.  [PATCH 1/3] Terminology: Device and driver, not host and guest.

    Posted 11-26-2013 03:13
    We've mixed both together, whereas from a spec-reader point of view, "driver" and "device" is probably clearer. I avoided changing the CCW part of the spec, since the guest/host language is probably more typical in that world. Signed-off-by: Rusty Russell <rusty@au.ibm.com> --- virtio-v1.0-wd01-part1-specification.txt 255 ++++++++++++++++--------------- 1 file changed, 134 insertions(+), 121 deletions(-) diff --git a/virtio-v1.0-wd01-part1-specification.txt b/virtio-v1.0-wd01-part1-specification.txt index 3be1bef..3ca180f 100644 --- a/virtio-v1.0-wd01-part1-specification.txt +++ b/virtio-v1.0-wd01-part1-specification.txt @@ -19,7 +19,7 @@ than boutique per-environment or per-OS mechanisms. Efficient: Virtio devices consist of rings of descriptors for input and output, which are neatly separated to avoid cache - effects from both guest and device writing to the same cache + effects from both driver and device writing to the same cache lines. Standard: Virtio makes no assumptions about the environment in which @@ -51,16 +51,29 @@ term -------------------- Guest - Definition... -Host - Definition + The entire system running inside the virtual machine. For + this specification, we are usually concerned with the guest + operating system which interacts with the devices. Device - Definition + + The device presented to the guest for its use. The device is + almost certainly "virtual", rather than a "real" hardware + device. Driver - Definition + + The software in the guest which interacts with the device. + This is typically the virtio-specific parts of the guest + operating system and a specific virtio driver within it. + +Host + + The outer system which operates the virtio device: typically a + hypervisor, other operating system or emulator. It has access + to the memory of the guest. + 1.2. Normative References ========================= @@ -98,7 +111,7 @@ To reinforce this the examples use typenames like "le16" instead of "uint16_t". 2.1.1. Device Status Field ------------------------- -The Device Status field is updated by the guest to indicate its +The Device Status field is updated by the OS and driver to indicate its progress. This provides a simple low-level diagnostic: it's most useful to imagine them hooked up to traffic lights on the console indicating the status of each device. @@ -119,7 +132,7 @@ This field is 0 upon reset, otherwise at least one bit should be set: DRIVER_OK (4) Indicates that the driver is set up and ready to drive the device. - FAILED (128) Indicates that something went wrong in the guest, + FAILED (128) Indicates that something went wrong in the guest OS, and it has given up on the device. This could be an internal error, or the driver didn't like the device for some reason, or even a fatal error during device operation. The device must be @@ -129,14 +142,14 @@ This field is 0 upon reset, otherwise at least one bit should be set: ------------------ Each virtio device lists all the features it understands. During -device initialization, the guest reads this and tells the device the +device initialization, the driver reads this and tells the device the subset that it understands. The only way to renegotiate is to reset the device. This allows for forwards and backwards compatibility: if the device is -enhanced with a new feature bit, older guests will not write that +enhanced with a new feature bit, older drivers will not write that feature bit back to the device and it can go into backwards -compatibility mode. Similarly, if a guest is enhanced with a feature +compatibility mode. Similarly, if a driver is enhanced with a feature that the device doesn't support, it see the new feature is not offered and can go into backwards compatibility mode (or, for poor implementations, set the FAILED Device Status bit). @@ -155,7 +168,7 @@ Device ID 1) indicates that the device supports checksumming of packets. In particular, new fields in the device configuration space are -indicated by offering a feature bit, so the guest can check +indicated by offering a feature bit, so the driver can check before accessing that part of the configuration space. 2.1.2.1 Legacy Interface: A Note on transitions from earlier drafts @@ -397,7 +410,7 @@ VIRTIO_F_ANY_LAYOUT feature is accepted. 2.1.4.3. The Virtqueue Descriptor Table -------------------------------------- -The descriptor table refers to the buffers the guest is using for +The descriptor table refers to the buffers the driver is using for the device. The addresses are physical addresses, and the buffers can be chained via the next field. Each descriptor describes a buffer which is read-only or write-only, but a chain of @@ -469,7 +482,7 @@ The available ring refers to what descriptor chains we are offering the device: each entry refers to the head of a descriptor chain. The “flags” field is currently 0 or 1: 1 indicating that we do not need an interrupt when the device consumes a descriptor chain from the available -ring. Alternatively, the guest can ask the device to delay interrupts +ring. Alternatively, the driver can ask the device to delay interrupts until an entry with an index specified by the “used_event” field is written in the used ring (equivalently, until the idx field in the used ring will reach the value used_event + 1). The method employed by @@ -493,21 +506,21 @@ entry (modulo the queue size). This starts at 0, and increases. The used ring is where the device returns buffers once it is done with them. The flags field can be used by the device to hint that -no notification is necessary when the guest adds to the available +no notification is necessary when the driver adds to the available ring. Alternatively, the “avail_event” field can be used by the device to hint that no notification is necessary until an entry with an index specified by the “avail_event” is written in the available ring (equivalently, until the idx field in the available ring will reach the value avail_event + 1). The method -employed by the device is controlled by the guest through the +employed by the device is controlled by the driver through the VIRTIO_RING_F_EVENT_IDX feature bit (see "2.6. Reserved Feature Bits").[7] Each entry in the ring is a pair: the head entry of the descriptor chain describing the buffer (this matches an entry -placed in the available ring by the guest earlier), and the total +placed in the available ring by the driver earlier), and the total of bytes written into the buffer. The latter is extremely useful -for guests using untrusted buffers: if you do not know exactly +for drivers using untrusted buffers: if you do not know exactly how much has been written by the device, you usually have to zero the buffer to ensure no data leakage occurs. @@ -570,7 +583,7 @@ how to communicate with the specific device. 8. The DRIVER_OK status bit is set. At this point the device is "live". -If any of these steps go irrecoverably wrong, the guest should +If any of these steps go irrecoverably wrong, the driver should set the FAILED status bit to indicate that it has given up on the device (it can reset the device later to restart if desired). @@ -757,11 +770,11 @@ buffer: at 65536 as well: (u16)(new_idx - used_event - 1) < (u16)(new_idx - old_idx) -For each ring, guest should then disable interrupts by writing +For each ring, driver should then disable interrupts by writing VRING_AVAIL_F_NO_INTERRUPT flag in avail structure, if required. It can then process used ring entries finally enabling interrupts by clearing the VRING_AVAIL_F_NO_INTERRUPT flag or updating the -EVENT_IDX field in the available structure. The guest should then +EVENT_IDX field in the available structure. The driver should then execute a memory barrier, and then recheck the ring empty condition. This is necessary to handle the case where after the last check and before enabling interrupts, an interrupt has been @@ -811,7 +824,7 @@ Any PCI device with Vendor ID 0x1AF4, and Device ID 0x1000 through The Subsystem Device ID indicates which virtio device is supported by the device. The Subsystem Vendor ID should reflect the PCI Vendor ID of the environment (it's currently only used -for informational purposes by the guest). +for informational purposes by the driver). All Drivers must match devices with any Revision ID, this is to allow devices to be versioned without breaking drivers. @@ -853,8 +866,8 @@ struct virtio_pci_common_cfg { /* About the whole device. */ le32 device_feature_select; /* read-write */ le32 device_feature; /* read-only */ - le32 guest_feature_select; /* read-write */ - le32 guest_feature; /* read-write */ + le32 driver_feature_select; /* read-write */ + le32 driver_feature; /* read-write */ le16 msix_config; /* read-write */ le16 num_queues; /* read-only */ u8 device_status; /* read-write */ @@ -880,22 +893,22 @@ device_feature_select device_feature - Used by Device to report Feature Bits to Driver. - Device Feature Bits selected by device_feature_select. + Used by device to report Feature Bits to Driver. + device Feature Bits selected by device_feature_select. -guest_feature_select +driver_feature_select - Selects which Feature Bits does guest_feature field refer to. + Selects which Feature Bits does driver_feature field refer to. Value 0x0 selects Feature Bits 0 to 31 Value 0x1 selects Feature Bits 32 to 63 - When set to any other value, reads from guest_feature - return 0, writing 0 into guest_feature has no effect, and - writing any other value into guest_feature is an error. + When set to any other value, reads from driver_feature + return 0, writing 0 into driver_feature has no effect, and + writing any other value into driver_feature is an error. -guest_feature +driver_feature - Used by Driver to acknowledge Feature Bits to Device. - Guest Feature Bits selected by guest_feature_select. + Used by Driver to acknowledge Feature Bits to device. + Driver Feature Bits selected by driver_feature_select. msix_config @@ -934,7 +947,7 @@ queue_msix_vector queue_enable - Used to selectively prevent host from executing requests from this virtqueue. + Used to selectively prevent device from executing requests from this virtqueue. 1 - enabled; 0 - disabled queue_notify_off @@ -1340,11 +1353,11 @@ If an interrupt is necessary: ii. If Queue Vector field value is NO_VECTOR, no interrupt message is requested for this event. -The guest interrupt handler should: +The driver interrupt handler should: 1. If MSI-X capability is disabled: read the ISR Status field, which will reset it to zero. If the lower bit is zero, the - interrupt was not for this device. Otherwise, the guest driver + interrupt was not for this device. Otherwise, the driver should look through the used rings of each virtqueue for the device, to see if any progress has been made by the device which requires servicing. @@ -1426,39 +1439,39 @@ configuration space. The following list presents their layout: * 0x00c R VendorID Virtio Subsystem Vendor ID. -* 0x010 R HostFeatures +* 0x010 R DeviceFeatures Flags representing features the device supports. Reading from this register returns 32 consecutive flag bits, first bit depending on the last value written to the - HostFeaturesSel register. Access to this register returns - bits HostFeaturesSel*32 to (HostFeaturesSel*32)+31, eg. - feature bits 0 to 31 if HostFeaturesSel is set to 0 and - features bits 32 to 63 if HostFeaturesSel is set to 1. + DeviceFeaturesSel register. Access to this register returns + bits DeviceFeaturesSel*32 to (DeviceFeaturesSel*32)+31, eg. + feature bits 0 to 31 if DeviceFeaturesSel is set to 0 and + features bits 32 to 63 if DeviceFeaturesSel is set to 1. Also see "2.1.2. Feature Bits". -* 0x014 W HostFeaturesSel - Device (Host) features word selection. +* 0x014 W DeviceFeaturesSel + Device (host) features word selection. Writing to this register selects a set of 32 device feature bits - accessible by reading from the HostFeatures register. Device driver - must write a value to the HostFeaturesSel register before - reading from the HostFeatures register. + accessible by reading from the DeviceFeatures register. The driver + must write a value to the DeviceFeaturesSel register before + reading from the DeviceFeatures register. -* 0x020 W GuestFeatures +* 0x020 W DriverFeatures Flags representing device features understood and activated by the driver. Writing to this register sets 32 consecutive flag bits, first - bit depending on the last value written to the GuestFeaturesSel - register. Access to this register sets bits GuestFeaturesSel*32 - to (GuestFeaturesSel*32)+31, eg. feature bits 0 to 31 if - GuestFeaturesSel is set to 0 and features bits 32 to 63 if - GuestFeaturesSel is set to 1. Also see "2.1.2. Feature Bits". + bit depending on the last value written to the DriverFeaturesSel + register. Access to this register sets bits DriverFeaturesSel*32 + to (DriverFeaturesSel*32)+31, eg. feature bits 0 to 31 if + DriverFeaturesSel is set to 0 and features bits 32 to 63 if + DriverFeaturesSel is set to 1. Also see "2.1.2. Feature Bits". -* 0x024 W GuestFeaturesSel +* 0x024 W DriverFeaturesSel Activated (Guest) features word selection. Writing to this register selects a set of 32 activated feature - bits accessible by writing to the GuestFeatures register. - Device driver must write a value to the GuestFeaturesSel - register before writing to the GuestFeatures register. + bits accessible by writing to the DriverFeatures register. + The driver must write a value to the DriverFeaturesSel + register before writing to the DriverFeatures register. * 0x030 W QueueSel Virtual queue index (first queue is 0). @@ -1470,7 +1483,7 @@ configuration space. The following list presents their layout: * 0x034 R QueueNumMax Maximum virtual queue size. Reading from the register returns the maximum size of the queue - the Host is ready to process or zero (0x0) if the queue is not + the device is ready to process or zero (0x0) if the queue is not available. This applies to the queue selected by writing to QueueSel and is allowed only when QueueReady is set to zero (0x0), so when the queue is not in use. @@ -1479,24 +1492,24 @@ configuration space. The following list presents their layout: Virtual queue size. Queue size is the number of elements in the queue, therefore size of the Descriptor Table and both Available and Used rings. - Writing to this register notifies the Host what size of the - queue the Guest will use. This applies to the queue selected by + Writing to this register notifies the device what size of the + queue the Driver will use. This applies to the queue selected by writing to QueueSel and is allowed only when QueueReady is set to zero (0x0), so when the queue is not in use. * 0x03c RW QueueReady Virtual queue ready bit. - Writing one (0x1) to this register notifies the Host that the + Writing one (0x1) to this register notifies the device that the virtual queue is ready to be used. Reading from this register returns the last value written to it. Both read and write accesses apply to the queue selected by writing to QueueSel. - When the Guest wants to stop using the queue it must write + When the Driver wants to stop using the queue it must write zero (0x0) to this register and read the value back to ensure synchronisation. * 0x050 W QueueNotify Queue notifier. - Writing a queue index to this register notifies the Host that + Writing a queue index to this register notifies the device that there are new buffers to process in the queue. * 0x60 R InterruptStatus @@ -1506,7 +1519,7 @@ configuration space. The following list presents their layout: corresponding bit is set, ie. equals one (1). – Bit 0 Used Ring Update - This interrupt is asserted when the Host has updated the Used + This interrupt is asserted when the device has updated the Used Ring in at least one of the active virtual queues. – Bit 1 Configuration change @@ -1515,7 +1528,7 @@ configuration space. The following list presents their layout: * 0x064 W InterruptACK Interrupt acknowledge. - Writing to this register notifies the Host that the Guest + Writing to this register notifies the device that the Driver finished handling interrupts. Set bits in the value clear the corresponding bits of the InterruptStatus register. @@ -1524,7 +1537,7 @@ configuration space. The following list presents their layout: Reading from this register returns the current device status flags. Writing non-zero values to this register sets the status flags, - indicating the Guest progress. Writing zero (0x0) to this + indicating the OS/driver progress. Writing zero (0x0) to this register triggers a device reset, including clearing all bits in the InterruptStatus register and ready bits in the QueueReady register for all queues in the device. @@ -1535,7 +1548,7 @@ configuration space. The following list presents their layout: Virtual queue's Descriptor Table 64 bit long physical address. Writing to these two registers (lower 32 bits of the address to QueueDescLow, higher 32 bits to QueueDescHigh) notifies - the host about location of the Descriptor Table of the queue + the device about location of the Descriptor Table of the queue selected by writing to the QueueSel register. It is allowed only when QueueReady is set to zero (0x0), so when the queue is not in use. @@ -1545,7 +1558,7 @@ configuration space. The following list presents their layout: Virtual queue's Available Ring 64 bit long physical address. Writing to these two registers (lower 32 bits of the address to QueueAvailLow, higher 32 bits to QueueAvailHigh) notifies - the host about location of the Available Ring of the queue + the device about location of the Available Ring of the queue selected by writing to the QueueSel register. It is allowed only when QueueReady is set to zero (0x0), so when the queue is not in use. @@ -1555,7 +1568,7 @@ configuration space. The following list presents their layout: Virtual queue's Used Ring 64 bit long physical address. Writing to these two registers (lower 32 bits of the address to QueueUsedLow, higher 32 bits to QueueUsedHigh) notifies - the host about location of the Used Ring of the queue + the device about location of the Used Ring of the queue selected by writing to the QueueSel register. It is allowed only when QueueReady is set to zero (0x0), so when the queue is not in use. @@ -1586,7 +1599,7 @@ can cause undefined behavior. 2.3.2.3.1. Device Initialization -------------------------------- -The guest must start the device initialization by reading and +The driver must start the device initialization by reading and checking values from the MagicValue and the Version registers. If both values are valid, it must read the DeviceID register and if its value is zero (0x0) must abort initialization and @@ -1612,9 +1625,9 @@ p. 2.2.1. "Device Initialization". is physically contiguous. It is recommended to align the Used Ring to an optimal boundary (usually page size). Size of the allocated queue may be smaller than or equal to - the maximum size returned by the Host. + the maximum size returned by the device. -5. Notify the Host about the queue size by writing the size to +5. Notify the device about the queue size by writing the size to the QueueNum register. 6. Write physical addresses of the queue's Descriptor Table, @@ -1681,9 +1694,9 @@ nor behaviour: * 0x028 W GuestPageSize Guest page size. - Device driver must write the guest page size in bytes to the + The driver must write the guest page size in bytes to the register during initialization, before any queues are used. - This value must be a power of 2 and is used by the Host to + This value must be a power of 2 and is used by the device to calculate the Guest address of the first queue page (see QueuePFN). @@ -1696,7 +1709,7 @@ nor behaviour: * 0x034 R QueueNumMax Maximum virtual queue size. Reading from the register returns the maximum size of the queue - the Host is ready to process or zero (0x0) if the queue is not + the device is ready to process or zero (0x0) if the queue is not available. This applies to the queue selected by writing to the QueueSel and is allowed only when the QueuePFN is set to zero (0x0), so when the queue is not actively used. @@ -1705,24 +1718,24 @@ nor behaviour: Virtual queue size. Queue size is the number of elements in the queue, therefore size of the descriptor table and both available and used rings. - Writing to this register notifies the Host what size of the - queue the Guest will use. This applies to the queue selected by + Writing to this register notifies the device what size of the + queue the Driver will use. This applies to the queue selected by writing to the QueueSel register. * 0x03c W QueueAlign Used Ring alignment in the virtual queue. - Writing to this register notifies the Host about alignment + Writing to this register notifies the device about alignment boundary of the Used Ring in bytes. This value must be a power of 2 and applies to the queue selected by writing to the QueueSel register. * 0x040 RW QueuePFN Guest physical page number of the virtual queue. - Writing to this register notifies the host about location of the + Writing to this register notifies the device about location of the virtual queue in the Guest's physical address space. This value is the index number of a page starting with the queue Descriptor Table. Value zero (0x0) means physical address zero - (0x00000000) and is illegal. When the Guest stops using the + (0x00000000) and is illegal. When the Driver stops using the queue it must write zero (0x0) to this register. Reading from this register returns the currently used page number of the queue, therefore a value other than zero (0x0) @@ -1741,7 +1754,7 @@ nor behaviour: Reading from this register returns the current device status flags. Writing non-zero values to this register sets the status flags, - indicating the Guest progress. Writing zero (0x0) to this + indicating the OS/driver progress. Writing zero (0x0) to this register triggers a device reset. This should include setting QueuePFN to zero (0x0) for all queues in the device. Also see "2.2.1. Device Initialization". @@ -1771,12 +1784,12 @@ The virtual queue is configured as follows: 4. Allocate and zero the queue pages in contiguous virtual memory, aligning the Used Ring to an optimal boundary (usually page size). Size of the allocated queue may be smaller than or - equal to the maximum size returned by the Host. + equal to the maximum size returned by the device. -5. Notify the Host about the queue size by writing the size to +5. Notify the device about the queue size by writing the size to the QueueNum register. -6. Notify the Host about the used alignment by writing its value +6. Notify the device about the used alignment by writing its value in bytes to the QueueAlign register. 7. Write the physical number of the first page of the queue to @@ -2318,20 +2331,20 @@ features. VIRTIO_NET_F_CSUM (0) Device handles packets with partial checksum - VIRTIO_NET_F_GUEST_CSUM (1) Guest handles packets with partial checksum + VIRTIO_NET_F_GUEST_CSUM (1) Driver handles packets with partial checksum VIRTIO_NET_F_CTRL_GUEST_OFFLOADS (2) Control channel offloads reconfiguration support. VIRTIO_NET_F_MAC (5) Device has given MAC address. - VIRTIO_NET_F_GUEST_TSO4 (7) Guest can receive TSOv4. + VIRTIO_NET_F_GUEST_TSO4 (7) Driver can receive TSOv4. - VIRTIO_NET_F_GUEST_TSO6 (8) Guest can receive TSOv6. + VIRTIO_NET_F_GUEST_TSO6 (8) Driver can receive TSOv6. - VIRTIO_NET_F_GUEST_ECN (9) Guest can receive TSO with ECN. + VIRTIO_NET_F_GUEST_ECN (9) Driver can receive TSO with ECN. - VIRTIO_NET_F_GUEST_UFO (10) Guest can receive UFO. + VIRTIO_NET_F_GUEST_UFO (10) Driver can receive UFO. VIRTIO_NET_F_HOST_TSO4 (11) Device can receive TSOv4. @@ -2341,7 +2354,7 @@ features. VIRTIO_NET_F_HOST_UFO (14) Device can receive UFO. - VIRTIO_NET_F_MRG_RXBUF (15) Guest can merge receive buffers. + VIRTIO_NET_F_MRG_RXBUF (15) Driver can merge receive buffers. VIRTIO_NET_F_STATUS (16) Configuration status field is available. @@ -2352,7 +2365,7 @@ features. VIRTIO_NET_F_CTRL_VLAN (19) Control channel VLAN filtering. - VIRTIO_NET_F_GUEST_ANNOUNCE(21) Guest can send gratuitous + VIRTIO_NET_F_GUEST_ANNOUNCE(21) Driver can send gratuitous packets. 2.4.1.3.1. Legacy Interface: Feature bits @@ -2395,7 +2408,7 @@ native endian of the guest rather than (necessarily) little-endian. 2. If the VIRTIO_NET_F_MAC feature bit is set, the configuration space “mac” entry indicates the “physical” address of the the network card, otherwise a private MAC address should be - assigned. All guests are expected to negotiate this feature if + assigned. All drivers are expected to negotiate this feature if it is set. 3. If the VIRTIO_NET_F_CTRL_VQ feature bit is negotiated, @@ -2527,7 +2540,7 @@ fully populated as possible: if it runs out, network performance will suffer. If the VIRTIO_NET_F_GUEST_TSO4, VIRTIO_NET_F_GUEST_TSO6 or -VIRTIO_NET_F_GUEST_UFO features are used, the Guest will need to +VIRTIO_NET_F_GUEST_UFO features are used, the Driver will need to accept packets of up to 65550 bytes long (the maximum size of a TCP or UDP packet, plus the 14 byte ethernet header), otherwise 1514. bytes. So unless VIRTIO_NET_F_MRG_RXBUF is negotiated, every @@ -2664,7 +2677,7 @@ native endian of the guest rather than (necessarily) little-endian. ------------------------------------- If the driver negotiates the VIRTIO_NET_F_GUEST_ANNOUNCE (depends -on VIRTIO_NET_F_CTRL_VQ), it can ask the guest to send gratuitous +on VIRTIO_NET_F_CTRL_VQ), it can ask the driver to send gratuitous packets; this is usually done after the guest has been physically migrated, and needs to announce its presence on the new network links. (As hypervisor does not have the knowledge of guest @@ -2674,7 +2687,7 @@ the guest in this way). #define VIRTIO_NET_CTRL_ANNOUNCE 3 #define VIRTIO_NET_CTRL_ANNOUNCE_ACK 0 -The Guest needs to check VIRTIO_NET_S_ANNOUNCE bit in status +The Driver needs to check VIRTIO_NET_S_ANNOUNCE bit in status field when it notices the changes of device configuration. The command VIRTIO_NET_CTRL_ANNOUNCE_ACK is used to indicate that driver has recevied the notification and device would clear the @@ -2767,7 +2780,7 @@ device except where noted. 2.4.2.3.1 Legacy Interface: Feature bits -------------------- - VIRTIO_BLK_F_BARRIER (0) Host supports request barriers. + VIRTIO_BLK_F_BARRIER (0) Device supports request barriers. VIRTIO_BLK_F_SCSI (7) Device supports scsi packet commands. @@ -2851,7 +2864,7 @@ negotiating VIRTIO_BLK_F_FLUSH: the cache will be in writeback mode after reset if and only if VIRTIO_BLK_F_FLUSH is negotiated. Some older legacy devices did not operate in writethrough mode even -after a guest announced lack of support for VIRTIO_BLK_F_FLUSH. +after a driver announced lack of support for VIRTIO_BLK_F_FLUSH. 2.4.2.5. Device Operation ------------------------ @@ -2881,8 +2894,8 @@ the read or write is to occur. This field is unused and set to 0 for scsi packet commands and for flush commands. The final status byte is written by the device: either -VIRTIO_BLK_S_OK for success, VIRTIO_BLK_S_IOERR for host or guest -error or VIRTIO_BLK_S_UNSUPP for a request unsupported by host: +VIRTIO_BLK_S_OK for success, VIRTIO_BLK_S_IOERR for device or driver +error or VIRTIO_BLK_S_UNSUPP for a request unsupported by device: #define VIRTIO_BLK_S_OK 0 #define VIRTIO_BLK_S_IOERR 1 @@ -2981,9 +2994,9 @@ of input and output virtqueues. Moreover, a device has a pair of control IO virtqueues. The control virtqueues are used to communicate information between the device and the driver about ports being opened and closed on either side of the connection, -indication from the host about whether a particular port is a +indication from the device about whether a particular port is a console port, adding new ports, port hot-plug/unplug, etc., and -indication from the guest about whether a port or a device was +indication from the driver about whether a port or a device was successfully added, port open/close, etc.. For data IO, one or more empty buffers are placed in the receive queue for incoming data and outgoing characters are placed in the transmit queue. @@ -3043,11 +3056,11 @@ native endian of the guest rather than (necessarily) little-endian. case, the control virtqueues are enabled and according to the max_nr_ports configuration-space value, the appropriate number of virtqueues are created. A control message indicating the - driver is ready is sent to the host. The host can then send + driver is ready is sent to the device. The device can then send control messages for adding new ports to the device. After creating and initializing each port, a - VIRTIO_CONSOLE_PORT_READY control message is sent to the host - for that port so the host can let us know of any additional + VIRTIO_CONSOLE_PORT_READY control message is sent to the device + for that port so the device can let us know of any additional configuration options set for that port. 3. The receiveq for each port is populated with one or more @@ -3068,14 +3081,14 @@ native endian of the guest rather than (necessarily) little-endian. be read from the configuration fields. 4. If the driver negotiated the VIRTIO_CONSOLE_F_MULTIPORT - feature, active ports are announced by the host using the + feature, active ports are announced by the device using the VIRTIO_CONSOLE_PORT_ADD control message. The same message is used for port hot-plug as well. -5. If the host specified a port `name', a sysfs attribute is +5. If the device specified a port `name', a sysfs attribute is created with the name filled in, so that udev rules can be written that can create a symlink from the port's name to the - char device for port discovery by applications in the guest. + char device for port discovery by applications in the driver. 6. Changes to ports' state are effected by control messages. Appropriate action is taken on the port indicated in the @@ -3143,7 +3156,7 @@ by random data by the device. The virtio memory balloon device is a primitive device for managing guest memory: the device asks for a certain amount of -memory, and the guest supplies it (or withdraws it, if the device +memory, and the driver supplies it (or withdraws it, if the device has more than it asks for). This allows the guest to adapt to changes in allowance of underlying physical memory. If the feature is negotiated, the device can also be used to communicate @@ -3192,7 +3205,7 @@ that legacy device fields are guest endian. (a) Identify the stats virtqueue. (b) Add one empty buffer to the stats virtqueue and notify the - host. + device. Device operation begins immediately. @@ -3345,7 +3358,7 @@ targets that receive and process the requests. ----------------------------------- All fields of this configuration are always available. sense_size - and cdb_size are writable by the guest. + and cdb_size are writable by the driver. struct virtio_scsi_config { le32 num_queues; @@ -3368,10 +3381,10 @@ targets that receive and process the requests. command. A bidirectional command can include seg_max input segments and seg_max output segments. - max_sectors is a hint to the guest about the maximum transfer + max_sectors is a hint to the driver about the maximum transfer size it should use. - cmd_per_lun is a hint to the guest about the maximum number of + cmd_per_lun is a hint to the driver about the maximum number of linked commands it should send to one LUN. The actual value to be used is the minimum of cmd_per_lun and the virtqueue size. @@ -3529,7 +3542,7 @@ following: (severed link). VIRTIO_SCSI_S_TARGET_FAILURE if the target is suffering a - failure and the guest should not retry on other paths. + failure and the driver should not retry on other paths. VIRTIO_SCSI_S_NEXUS_FAILURE if the nexus is suffering a failure but retrying on other paths might yield a different result. @@ -3537,7 +3550,7 @@ following: VIRTIO_SCSI_S_BUSY if the request failed but retrying on the same path should work. - VIRTIO_SCSI_S_FAILURE for other host or guest error. In + VIRTIO_SCSI_S_FAILURE for other host or driver error. In particular, if neither dataout nor datain is empty, and the VIRTIO_SCSI_F_INOUT feature has not been negotiated, the request will be immediately returned with a response equal to @@ -3779,8 +3792,8 @@ contents of the event field. The following events are defined: should ask the initiator to rescan the target, in order to detect the case when an entire target has appeared or disappeared. These two events will never be reported unless the - VIRTIO_SCSI_F_HOTPLUG feature was negotiated between the host - and the guest. + VIRTIO_SCSI_F_HOTPLUG feature was negotiated between the device + and the driver. Events will also be reported via sense codes (this obviously does not apply to newly appeared buses or targets, since the @@ -3857,7 +3870,7 @@ Currently there are four device-independent feature bits defined: and the avail_event fields. If set, it indicates that the device should ignore the flags field in the available ring structure. Instead, the used_event field in this structure is - used by guest to suppress device interrupts. Further, the + used by driver to suppress device interrupts. Further, the driver should ignore the flags field in the used ring structure. Instead, the avail_event field in this structure is used by the device to suppress notifications. If unset, the @@ -3938,11 +3951,11 @@ VIRTIO_F_ANY_LAYOUT (27) This feature indicates that the device /* This means the buffer contains a list of buffer descriptors. */ #define VRING_DESC_F_INDIRECT 4 -/* The Host uses this in used->flags to advise the Guest: don't kick me +/* The Device uses this in used->flags to advise the Driver: don't kick me * when you add a buffer. It's unreliable, so it's simply an - * optimization. Guest will still kick if it's out of buffers. */ + * optimization. Driver will still kick if it's out of buffers. */ #define VRING_USED_F_NO_NOTIFY 1 -/* The Guest uses this in avail->flags to advise the Host: don't +/* The Driver uses this in avail->flags to advise the Device: don't * interrupt me when you consume a buffer. It's unreliable, so it's * simply an optimization. */ #define VRING_AVAIL_F_NO_INTERRUPT 1 @@ -4111,7 +4124,7 @@ reducing interrupt sharing between devices and queues within the device, and handling interrupts from multiple CPUs. However, some systems impose a limit (which might be as low as 256) on the total number of MSI-X vectors that can be allocated to all -devices. Devices and/or device drivers should take this into +devices. Devices and/or drivers should take this into account, limiting the number of vectors used unless the device is expected to cause a high volume of interrupts. Devices can control the number of vectors used by limiting the MSI-X Table -- 1.8.3.2


  • 2.  [PATCH 3/3] Specify requirements more clearly.

    Posted 11-26-2013 03:13
    The spec language is mostly written with a view to driver authors, and contains assumptions, eg: The Device Status field is updated by the OS and driver... But the spec is for both device and driver authors. It should specify exactly what is to be done, and by whom, eg: The driver MUST update the Device Status field... 1) Change from passive to active (eg "foo must be reset" => "driver must reset foo"). 2) Upcase SHOULD, MUST etc. There are probably more that I missed. 3) Make the following requirements explicit: - The driver must not clear device status bits. - Make bad on looped descriptors explicit. - Driver must not set VRING_DESC_F_INDIRECT unless negotiated feature. - Available/used ring sections rewritten to be more explicit that driver must not set VRING_AVAIL_F_NO_INTERRUPT. - Following device initialization sequence is a MUST. - Driver must not continue initialization if it sets FAILED. - "memory barriers" are now a MUST, though the weasel-word "suitable" was added. - Driver MUST notify device. 4) Misc changes: - Use "device offers" / "driver accepts" language for feature negotiation. - config space always uses little-endian, remove 'generally'. - "descriptor chain" term used everywhere. - Extraneous "+" deleted. - Remove "Unless explicitly specified otherwise" from PCI spec endian sentence. - Refer to notify_off_multiplier in queue_notify_off discussion. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> --- virtio-v1.0-wd01-part1-specification.txt 321 ++++++++++++++++++------------- 1 file changed, 182 insertions(+), 139 deletions(-) diff --git a/virtio-v1.0-wd01-part1-specification.txt b/virtio-v1.0-wd01-part1-specification.txt index 52adf24..4e479c8 100644 --- a/virtio-v1.0-wd01-part1-specification.txt +++ b/virtio-v1.0-wd01-part1-specification.txt @@ -111,10 +111,11 @@ To reinforce this the examples use typenames like "le16" instead of "uint16_t". 2.1.1. Device Status Field ------------------------- -The Device Status field is updated by the OS and driver to indicate its -progress. This provides a simple low-level diagnostic: it's most -useful to imagine them hooked up to traffic lights on the console -indicating the status of each device. +The driver MUST update the Device Status field in the order below to +indicate its progress. This provides a simple low-level diagnostic: +it's most useful to imagine them hooked up to traffic lights on the +console indicating the status of each device. The driver MUST NOT +clear a device status bit. This field is 0 upon reset, otherwise at least one bit should be set: @@ -135,24 +136,31 @@ This field is 0 upon reset, otherwise at least one bit should be set: FAILED (128) Indicates that something went wrong in the guest OS, and it has given up on the device. This could be an internal error, or the driver didn't like the device for some reason, or - even a fatal error during device operation. The device must be - reset before attempting to re-initialize. + even a fatal error during device operation. The driver MUST + reset the device before attempting to re-initialize. 2.1.2. Feature Bits ------------------ -Each virtio device lists all the features it understands. During +Each virtio device offers all the features it understands. During device initialization, the driver reads this and tells the device the -subset that it understands. The only way to renegotiate is to reset +subset that it accepts. The only way to renegotiate is to reset the device. This allows for forwards and backwards compatibility: if the device is enhanced with a new feature bit, older drivers will not write that -feature bit back to the device and it can go into backwards +feature bit back to the device and it SHOULD go into backwards compatibility mode. Similarly, if a driver is enhanced with a feature that the device doesn't support, it see the new feature is not offered -and can go into backwards compatibility mode (or, for poor -implementations, set the FAILED Device Status bit). +and SHOULD go into backwards compatibility mode (or, for poor +implementations it MAY set the FAILED Device Status bit). + +The driver MUST NOT accept a feature which the device did not offer, +and MUST NOT accept a feature which requires another feature which was +not accepted. + +The device MUST NOT offer a feature which requires another feature +which was not offered. Feature bits are allocated as follows: @@ -168,8 +176,9 @@ Device ID 1) indicates that the device supports checksumming of packets. In particular, new fields in the device configuration space are -indicated by offering a feature bit, so the driver can check -before accessing that part of the configuration space. +indicated by offering a feature bit, so the driver MUST check that the +feature is offered before accessing that part of the configuration +space. 2.1.2.1 Legacy Interface: A Note on transitions from earlier drafts -------------------------------------- @@ -224,7 +233,7 @@ Interface' in the section title. ------------------------- Configuration space is generally used for rarely-changing or -initialization-time parameters. Drivers must not assume reads from +initialization-time parameters. Drivers MUST NOT assume reads from fields greater than 32 bits wide are atomic, nor or reads from multiple fields. @@ -233,7 +242,7 @@ space, which must change whenever there is a possibility that two accesses to the configuration space can see different versions of that space. -Thus drivers should read configuration space fields like so: +Thus drivers SHOULD read configuration space fields like so: u32 before, after; do { @@ -242,21 +251,21 @@ Thus drivers should read configuration space fields like so: after = get_config_generation(device); } while (after != before); -Note that configuration space generally uses the little-endian format +Note that configuration space uses the little-endian format for multi-byte fields. Note that future versions of this specification will likely extend the configuration space for devices by adding extra fields at the tail end of some structures in configuration space. -To allow forward compatibility with such extensions, drivers must -not limit structure size and configuration space size. Instead, -drivers should only check that configuration space is *large enough* to +To allow forward compatibility with such extensions, drivers MUST +NOT limit structure size and configuration space size. Instead, +drivers SHOULD only check that configuration space is *large enough* to contain the fields required for device operation. For example, if the specification states that configuration space 'includes a single 8-bit field' drivers should understand this to mean that -the configuration space can also include an arbitrary amount of +the configuration space might also include an arbitrary amount of tail padding, and accept any configuration space size equal to or greater than the specified 8-bit size. @@ -309,8 +318,8 @@ virtqueue are summarized in the following table: +------------+-----------------------------------------+ The Alignment column gives the miminum alignment: for each part -of the virtqueue, the physical address of the first byte of it -must be a multiple of the specified alignment value. +of the virtqueue, the physical address of the first byte +MUST be a multiple of the specified alignment value. The Size column gives the total number of bytes required for each part of the virtqueue. @@ -325,7 +334,7 @@ When the driver wants to send a buffer to the device, it fills in a slot in the descriptor table (or chains several together), and writes the descriptor index into the available ring. It then notifies the device. When the device has finished a buffer, it -writes the descriptor into the used ring, and sends an interrupt. +writes the descriptor index into the used ring, and sends an interrupt. 2.1.4.1. Legacy Interfaces: A Note on Virtqueue Layout @@ -378,7 +387,8 @@ It is assumed that the host is already aware of the guest endian. 2.1.4.2. Message Framing ----------------------- -The message framing (the particular layout of descriptors) is +The device MUST NOT make assumptions about the particular arrangement +of descriptors: the message framing is independent of the contents of the buffers. For example, a network transmit buffer consists of a 12 byte header followed by the network packet. This could be most simply placed in the descriptor table as a @@ -421,7 +431,8 @@ device type. Most common is to begin the data with a header (containing little-endian fields) for the device to read, and postfix it with a status tailer for the device to write. -No descriptor chain may be more than 2^32 bytes long in total. +Drivers MUST NOT add a descriptor chain over than 2^32 bytes long in total; +this implies that loops in the descriptor chain are forbidden! struct vring_desc { /* Address (guest-physical). */ @@ -448,14 +459,18 @@ for this virtqueue. ------------------------------ Some devices benefit by concurrently dispatching a large number -of large requests. The VIRTIO_RING_F_INDIRECT_DESC feature can be -used to allow this (see "2.6. Reserved Feature Bits"). To increase -ring capacity it is possible to store a table of indirect +of large requests. The VIRTIO_RING_F_INDIRECT_DESC feature allows this (see "2.6. Reserved Feature Bits"). To increase +ring capacity the driver can store a table of indirect descriptors anywhere in memory, and insert a descriptor in main virtqueue (with flags&VRING_DESC_F_INDIRECT on) that refers to memory buffer containing this indirect descriptor table; fields addr and len refer to the indirect table address and length in bytes, -respectively. The indirect table layout structure looks like this +respectively. + +The driver MUST NOT set the VRING_DESC_F_INDIRECT flag unless the +VIRTIO_RING_F_INDIRECT_DESC feature was negotiated. + +The indirect table layout structure looks like this (len is the length of the descriptor that refers to this table, which is a variable, so this code won't compile): @@ -470,29 +485,13 @@ chained by next field. An indirect descriptor without next field (with flags&VRING_DESC_F_NEXT off) signals the end of the descriptor. An indirect descriptor can not refer to another indirect descriptor -table (flags&VRING_DESC_F_INDIRECT must be off). A single indirect descriptor +table (flags&VRING_DESC_F_INDIRECT MUST be off). A single indirect descriptor table can include both read-only and write-only descriptors; -write-only flag (flags&VRING_DESC_F_WRITE) in the descriptor that refers to it -is ignored. +the device MUST ignore the write-only flag (flags&VRING_DESC_F_WRITE) in the descriptor that refers to it. 2.1.4.4. The Virtqueue Available Ring ------------------------------------ -The available ring refers to what descriptor chains we are offering the -device: each entry refers to the head of a descriptor chain. The “flags” field -is currently 0 or 1: 1 indicating that we do not need an interrupt -when the device consumes a descriptor chain from the available -ring. Alternatively, the driver can ask the device to delay interrupts -until an entry with an index specified by the “used_event” field is -written in the used ring (equivalently, until the idx field in the -used ring will reach the value used_event + 1). The method employed by -the device is controlled by the VIRTIO_RING_F_EVENT_IDX feature bit -(see "2.6. Reserved Feature Bits"). This interrupt suppression is -merely an optimization; it may not suppress interrupts entirely. - -The “idx” field indicates where we would put the next descriptor -entry (modulo the queue size). This starts at 0, and increases. - struct vring_avail { #define VRING_AVAIL_F_NO_INTERRUPT 1 le16 flags; @@ -501,28 +500,40 @@ entry (modulo the queue size). This starts at 0, and increases. le16 used_event; /* Only if VIRTIO_RING_F_EVENT_IDX */ }; +The available ring refers to what descriptor chains the driver is offering the +device: each ring entry refers to the head of a descriptor chain. It is only +written by the driver and read by the device. + +The “idx” field indicates where we would put the next descriptor +entry in the ring (modulo the queue size). This starts at 0, and increases. + +If the VIRTIO_RING_F_INDIRECT_DESC feature bit is not negotiated, the +“flags” field offers a crude interrupt control mechanism. The driver +MUST set this to 0 or 1: 1 indicates that the device SHOULD NOT send +an interrupt when it consumes a descriptor chain from the available +ring. The device MUST ignore the used_event value in this case. + +Otherwise, if the VIRTIO_RING_F_EVENT_IDX feature bit is negotiated, +the driver MUST set the "flags" field to 0, and use the “used_event” +field in the used ring instead. The driver can ask the device to delay interrupts +until an entry with an index specified by the “used_event” field is +written in the used ring (equivalently, until the idx field in the +used ring will reach the value used_event + 1). + +The driver MUST handle spurious interrupts: either form of interrupt +suppression is merely an optimization; it may not suppress interrupts +entirely. + 2.1.4.5. The Virtqueue Used Ring ------------------------------- -The used ring is where the device returns buffers once it is done -with them. The flags field can be used by the device to hint that -no notification is necessary when the driver adds to the available -ring. Alternatively, the “avail_event” field can be used by the -device to hint that no notification is necessary until an entry -with an index specified by the “avail_event” is written in the -available ring (equivalently, until the idx field in the -available ring will reach the value avail_event + 1). The method -employed by the device is controlled by the driver through the -VIRTIO_RING_F_EVENT_IDX feature bit (see "2.6. Reserved -Feature Bits").[7] - -Each entry in the ring is a pair: the head entry of the -descriptor chain describing the buffer (this matches an entry -placed in the available ring by the driver earlier), and the total -of bytes written into the buffer. The latter is extremely useful -for drivers using untrusted buffers: if you do not know exactly -how much has been written by the device, you usually have to zero -the buffer to ensure no data leakage occurs. + struct vring_used { + #define VRING_USED_F_NO_NOTIFY 1 + le16 flags; + le16 idx; + struct vring_used_elem ring[ /* Queue Size */]; + le16 avail_event; /* Only if VIRTIO_RING_F_EVENT_IDX */ + }; /* le32 is used here for ids for padding reasons. */ struct vring_used_elem { @@ -532,13 +543,35 @@ the buffer to ensure no data leakage occurs. le32 len; }; - struct vring_used { - #define VRING_USED_F_NO_NOTIFY 1 - le16 flags; - le16 idx; - struct vring_used_elem ring[ /* Queue Size */]; - le16 avail_event; /* Only if VIRTIO_RING_F_EVENT_IDX */ - }; +The used ring is where the device returns buffers once it is done with +them: it is only written to by the device, and read by the driver. + +Each entry in the ring is a pair: the head entry of the +descriptor chain describing the buffer (this matches an entry +placed in the available ring by the guest earlier), and the total +of bytes written into the buffer. The latter is extremely useful +for drivers using untrusted buffers: if you do not know exactly +how much has been written by the device, you usually have to zero +the buffer to ensure no data leakage occurs. + +If the VIRTIO_RING_F_INDIRECT_DESC feature bit is not negotiated, the +“flags” field offers a crude interrupt control mechanism. The driver +MUST initialize this to 0, the device MUST set this to 0 or 1: 1 +indicates that the driver SHOULD NOT send an notification when it adds +a descriptor chain to the available ring. The driver MUST ignore the +used_event value in this case. + +Otherwise, if the VIRTIO_RING_F_EVENT_IDX feature bit is negotiated, +the device MUST leave the "flags" field at 0, and use the +“avail_event” field in the used ring instead. The device can ask the +driver to delay notifications until an entry with an index specified +by the “avail_event” field is written in the available ring (equivalently, +until the idx field in the used ring will reach the value avail_event + +1). + +The device MUST handle spurious notification: either form of +notification suppression is merely an optimization; it may not +suppress them entirely. 2.1.4.6. Helpers for Operating Virtqueues ---------------------------------------- @@ -561,36 +594,40 @@ how to communicate with the specific device. 2.2.1. Device Initialization --------------------------- +The driver MUST follow this sequence to initialize a device: + 1. Reset the device. -2. The ACKNOWLEDGE status bit is set: we have noticed the device. +2. Set the ACKNOWLEDGE status bit: we have noticed the device. -3. The DRIVER status bit is set: we know how to drive the device. +3. Set the DRIVER status bit: we know how to drive the device. -4. Device feature bits are read, and the the subset of feature bits - understood by the OS and driver is written to the device. +4. Read device feature bits, and write the subset of feature bits + understood by the OS and driver to the device. -5. The FEATURES_OK status bit is set. +5. Set the FEATURES_OK status bit. The driver MUST not accept + new feature bits after this step. -6. The status byte is re-read to ensure the FEATURES_OK bit is still +6. Re-read the status byte to ensure the FEATURES_OK bit is still set: otherwise, the device does not support our subset of features and the device is unusable. -7. Device-specific setup, including discovery of virtqueues for the +7. Perform device-specific setup, including discovery of virtqueues for the device, optional per-bus setup, reading and possibly writing the device's virtio configuration space, and population of virtqueues. -8. The DRIVER_OK status bit is set. At this point the device is +8. Set the DRIVER_OK status bit. At this point the device is "live". -If any of these steps go irrecoverably wrong, the driver should +If any of these steps go irrecoverably wrong, the driver SHOULD set the FAILED status bit to indicate that it has given up on the -device (it can reset the device later to restart if desired). +device (it can reset the device later to restart if desired). The +driver MUST not continue initialization in that case. -The device must not consume buffers before DRIVER_OK, and the driver -must not notify the device before it sets DRIVER_OK. +The device MUST NOT consume buffers before DRIVER_OK, and the driver +MUST NOT notify the device before it sets DRIVER_OK. -Devices should support all valid combinations of features, but we know +Devices SHOULD support all valid combinations of features, but we know that implementations may well make assuptions that they will only be used by fully-optimized drivers. The resetting of the FEATURES_OK flag provides a semi-graceful failure mode for this case. @@ -625,29 +662,29 @@ they are used. 2.2.2.1. Supplying Buffers to The Device --------------------------------------- -Actual transfer of buffers from the guest OS to the device -operates as follows: +The driver offers buffers to one of the device's virtqueues as follows: -1. Place the buffer(s) into free descriptor(s). +1. The driver places the buffer into free descriptor(s) in the + descriptor table, chaining as necessary (see 2.1.4.3.). -2. Place the id of the buffer in the next ring entry of the - available ring. +2. The driver places the index of the head of the descriptor chain + into the next ring entry of the available ring. -3. The steps (1) and (2) may be performed repeatedly if batching +3. Steps (1) and (2) may be performed repeatedly if batching is possible. -4. A memory barrier should be executed to ensure the device sees +4. The driver MUST perform suitable a memory barrier to ensure the device sees the updated descriptor table and available ring before the next step. -5. The available “idx” field should be increased by the number of - entries added to the available ring. +5. The available “idx” field is increased by the number of + descriptor chain heads added to the available ring. -6. A memory barrier should be executed to ensure that we update - the idx field before checking for notification suppression. +6. The driver MUST perform a suitable memory barrier to ensure that it updates + the "idx" field before checking for notification suppression. -7. If notifications are not suppressed, the device should be - notified of the new buffers. +7. If notifications are not suppressed, the driver MUST notify the device + of the new available buffers. Note that the above code does not take precautions against the available ring buffer wrapping around: this is not possible since @@ -666,7 +703,8 @@ Here is a description of each stage in more detail. A buffer consists of zero or more read-only physically-contiguous elements followed by zero or more physically-contiguous write-only elements (it must have at least one element). This -algorithm maps it into the descriptor table: +algorithm maps it into the descriptor table to form a descriptor +chain: for each buffer element, b: @@ -694,7 +732,7 @@ free descriptors before beginning the mappings. ------------------------------------- The head of the buffer we mapped is the first d in the algorithm -above. A naive implementation would do the following (with the +above (the descriptor chain head). A naive implementation would do the following (with the appropriate conversion to-and-from little-endian assumed): avail->ring[avail->idx % qsz] = head; @@ -724,7 +762,7 @@ The index field always increments, and we let it wrap naturally at The actual method of device notification is bus-specific, but generally it can be expensive. So the device can suppress such notifications if it -doesn't need them. We have to be careful to expose the new index +doesn't need them. The driver has to be careful to expose the new index value before checking if notifications are suppressed: it's OK to notify gratuitously, but not to omit a required notification. So again, we use a memory barrier here before reading the flags or the @@ -822,11 +860,11 @@ Any PCI device with Vendor ID 0x1AF4, and Device ID 0x1000 through 0x103F inclusive is a virtio device[3]. The Subsystem Device ID indicates which virtio device is -supported by the device. The Subsystem Vendor ID should reflect +supported by the device. The Subsystem Vendor ID SHOULD reflect the PCI Vendor ID of the environment (it's currently only used for informational purposes by the driver). -All Drivers must match devices with any Revision ID, this +All drivers MUST match devices with any Revision ID, this is to allow devices to be versioned without breaking drivers. 2.3.1.1.1 Legacy Interfaces: A Note on PCI Device Discovery @@ -846,7 +884,7 @@ To configure the device, use I/O and/or memory regions and/or PCI configuration space of the PCI device. These contain the virtio header registers, the notification register, the ISR status register and device specific registers, as specified by Virtio -+ Structure PCI Capabilities +Structure PCI Capabilities. There may be different widths of accesses to the I/O region; the “natural” access method for each field must be @@ -856,7 +894,7 @@ PCI Device Configuration Layout includes the common configuration, ISR, notification and device specific configuration structures. -Unless explicitly specified otherwise, all multi-byte fields are little-endian. +All multi-byte fields are little-endian. 2.3.1.2.1. Common configuration structure layout ------------------------- @@ -886,87 +924,92 @@ struct virtio_pci_common_cfg { device_feature_select - Selects which Feature Bits does device_feature field refer to. + The driver uses this to select which Feature Bits the device_feature field shows. Value 0x0 selects Feature Bits 0 to 31 Value 0x1 selects Feature Bits 32 to 63 - All other values cause reads from device_feature to return 0. + The device MUST present 0 on device_feature for any other value. device_feature - Used by device to report Feature Bits to Driver. - device Feature Bits selected by device_feature_select. + The device uses this to report Feature Bits to the driver. + Device Feature Bits selected by device_feature_select. driver_feature_select - Selects which Feature Bits does driver_feature field refer to. + The driver uses this to select which Feature Bits the driver_feature field shows. Value 0x0 selects Feature Bits 0 to 31 Value 0x1 selects Feature Bits 32 to 63 When set to any other value, reads from driver_feature - return 0, writing 0 into driver_feature has no effect, and - writing any other value into driver_feature is an error. + return 0, writing 0 into driver_feature has no effect. The driver + MUST not write any other value into driver_feature (a corollary of + the rule that the driver can only write a subset of device features). driver_feature - Used by Driver to acknowledge Feature Bits to device. + The driver writes this to accept feature bits offered by the deevice. Driver Feature Bits selected by driver_feature_select. msix_config - Configuration Vector for MSI-X. + The driver sets the Configuration Vector for MSI-X. num_queues - Specifies the maximum number of virtqueues supported by device. + The device specifies the maximum number of virtqueues supported here. device_status - Device Status field. Writing 0 into this field resets the - device. + The driver writes the Device Status here. Writing 0 into this + field resets the device. config_generation - Configuration atomicity value. Changes every time the + Configuration atomicity value. The device changes this every time the configuration noticeably changes. This means the device may only change the value after a configuration read operation, - but it must change if there is any risk of a device seeing an + but MUST change it if there is any risk of a driver seeing an inconsistent configuration state. queue_select - Queue Select. Selects which virtqueue do other fields refer to. + Queue Select. The driver selects which virtqueue the following + fields refer to. queue_size Queue Size. On reset, specifies the maximum queue size supported by the hypervisor. This can be modified by driver to reduce memory requirements. - Set to 0 if this virtqueue is unused. + The device MUST set this to 0 if this virtqueue is unavailable. queue_msix_vector - Queue Vector for MSI-X. + The driver uses this to specify the Queue Vector for MSI-X. queue_enable - Used to selectively prevent device from executing requests from this virtqueue. + The driver uses this to selectively prevent the device from executing requests from this virtqueue. 1 - enabled; 0 - disabled + The driver MUST configure the other virtqueue fields before enabling + the virtqueue. + queue_notify_off - Used to calculate the offset from start of Notification structure at + The driver reads this to calculate the offset from start of Notification structure at which this virtqueue is located. Note: this is *not* an offset in bytes. See notify_off_multiplier below. queue_desc - Physical address of Descriptor Table. + The driver writes the physical address of Descriptor Table here. queue_avail - Physical address of Available Ring. + The driver writes the physical address of Available Ring here. queue_used - Physical address of Used Ring. + The driver writes the physical address of Used Ring here. 2.3.1.2.2. ISR status structure layout ------------------------- @@ -977,7 +1020,7 @@ ISR status structure includes a single 8-bit ISR status field. Notification structure is always a multiple of 2 bytes in size. It includes 2-byte Queue Notify fields for each virtqueue of the device. Note that multiple virtqueues can use the same -Queue Notify field, if necessary. +Queue Notify field, if necessary: see notify_off_multiplier below. 2.3.1.2.4. Device specific structure ------------------------- @@ -1140,14 +1183,14 @@ cfg_type /* PCI configuration access */ #define VIRTIO_PCI_CAP_PCI_CFG 5 - Any other value - reserved for future use. Drivers must + Any other value - reserved for future use. Drivers MUST ignore any vendor-specific capability structure which has a reserved cfg_type value. More than one capability can identify the same structure - this makes it possible for the device to expose multiple interfaces to drivers. The order of the capabilities in the capability list specifies the order of preference - suggested by the device; drivers should use the first interface that they can + suggested by the device; drivers SHOULD use the first interface that they can support. For example, on some hypervisors, notifications using IO accesses are faster than memory accesses. In this case, hypervisor can expose two capabilities with cfg_type set to VIRTIO_PCI_CAP_NOTIFY_CFG: @@ -1162,7 +1205,7 @@ bar The BAR is permitted to be either 32-bit or 64-bit, it can map Memory Space or I/O Space. - Any other value - reserved for future use. Drivers must + Any other value is reserved for future use. Drivers MUST ignore any vendor-specific capability structure which has a reserved bar value. @@ -1173,7 +1216,7 @@ offset length indicates the length of the structure. This size might include padding, or fields unused by the driver. - Drivers are also recommended to only map part of configuration structure + Drivers SHOULD only map part of configuration structure large enough for device operation. For example, a future device might present a large structure size of several MBytes. @@ -1283,13 +1326,13 @@ or NO_VECTOR if unmapped. All queue and configuration change events are unmapped by default. Note that mapping an event to vector might require allocating -internal device resources, and might fail. Devices report such +internal device resources, and might fail. Devices MUST report such failures by returning the NO_VECTOR value when the relevant Vector field is read. After mapping an event to vector, the -driver must verify success by reading the Vector field value: on +driver MUST verify success by reading the Vector field value: on success, the previously written value is returned, and on failure, NO_VECTOR is returned. If a mapping failure is detected, -the driver can retry mapping with fewervectors, or disable MSI-X. +the driver can retry mapping with fewer vectors, or disable MSI-X. 2.3.1.3.1.2. Virtqueue Configuration ----------------------------------- @@ -1299,13 +1342,13 @@ transport (for example, the simplest network device has two), the driver needs to configure them as part of the device-specific configuration. -This is done as follows, for each virtqueue a device has: +The driver does this as follows, for each virtqueue a device has: 1. Write the virtqueue index (first queue is 0) to the Queue Select field. -2. Read the virtqueue size from the Queue Size field, which is - always a power of 2. This controls how big the virtqueue is +2. Read the virtqueue size from the Queue Size field, which MUST + be a power of 2. This controls how big the virtqueue is (see "2.1.4. Virtqueues"). If this field is 0, the virtqueue does not exist. 3. Optionally, select a smaller virtqueue size and write it in the Queue Size -- 1.8.3.2


  • 3.  [PATCH 2/3] pci: fix config notify bit.

    Posted 11-26-2013 03:13
    The Linux code uses 0x2, so I assume second lowest bit is right... Signed-off-by: Rusty Russell <rusty@au.ibm.com> --- virtio-v1.0-wd01-part1-specification.txt 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/virtio-v1.0-wd01-part1-specification.txt b/virtio-v1.0-wd01-part1-specification.txt index 3ca180f..52adf24 100644 --- a/virtio-v1.0-wd01-part1-specification.txt +++ b/virtio-v1.0-wd01-part1-specification.txt @@ -1375,7 +1375,7 @@ state, as reflected in the virtio header in the PCI configuration space. In this case: 1. If MSI-X capability is disabled: an interrupt is delivered and - the second highest bit is set in the ISR Status field to + the second lowest bit is set in the ISR Status field to indicate that the driver should re-examine the configuration space. Note that a single interrupt can indicate both that one or more virtqueue has been used and that the configuration -- 1.8.3.2


  • 4.  Re: [PATCH 1/3] Terminology: Device and driver, not host and guest.

    Posted 11-26-2013 16:29
    On Tue, 26 Nov 2013 13:41:13 +1030 Rusty Russell <rusty@au1.ibm.com> wrote: > We've mixed both together, whereas from a spec-reader point of view, > "driver" and "device" is probably clearer. > > I avoided changing the CCW part of the spec, since the guest/host > language is probably more typical in that world. OTOH, having different terminology for different transports might be confusing (and we do talk about devices presented via a control unit). The uncommon term is "driver"; s390 documentation usually talks about "the program". host/guest makes sense when talking about the whole picture as in host->guest notification. I'll talk to some folks here to get a second opinion on this. > > Signed-off-by: Rusty Russell <rusty@au.ibm.com> > --- > virtio-v1.0-wd01-part1-specification.txt 255 ++++++++++++++++--------------- > 1 file changed, 134 insertions(+), 121 deletions(-) > > diff --git a/virtio-v1.0-wd01-part1-specification.txt b/virtio-v1.0-wd01-part1-specification.txt > index 3be1bef..3ca180f 100644 > --- a/virtio-v1.0-wd01-part1-specification.txt > +++ b/virtio-v1.0-wd01-part1-specification.txt > @@ -98,7 +111,7 @@ To reinforce this the examples use typenames like "le16" instead of "uint16_t". > 2.1.1. Device Status Field > ------------------------- > > -The Device Status field is updated by the guest to indicate its > +The Device Status field is updated by the OS and driver to indicate its How about "guest/driver"? We haven't defined OS anywhere :) > progress. This provides a simple low-level diagnostic: it's most > useful to imagine them hooked up to traffic lights on the console > indicating the status of each device. > @@ -119,7 +132,7 @@ This field is 0 upon reset, otherwise at least one bit should be set: > DRIVER_OK (4) Indicates that the driver is set up and ready to > drive the device. > > - FAILED (128) Indicates that something went wrong in the guest, > + FAILED (128) Indicates that something went wrong in the guest OS, I'd keep this at "guest". > and it has given up on the device. This could be an internal > error, or the driver didn't like the device for some reason, or > even a fatal error during device operation. The device must be > @@ -757,11 +770,11 @@ buffer: > at 65536 as well: > (u16)(new_idx - used_event - 1) < (u16)(new_idx - old_idx) > > -For each ring, guest should then disable interrupts by writing > +For each ring, driver should then disable interrupts by writing s/driver/the driver/ > VRING_AVAIL_F_NO_INTERRUPT flag in avail structure, if required. > It can then process used ring entries finally enabling interrupts > by clearing the VRING_AVAIL_F_NO_INTERRUPT flag or updating the > -EVENT_IDX field in the available structure. The guest should then > +EVENT_IDX field in the available structure. The driver should then > execute a memory barrier, and then recheck the ring empty > condition. This is necessary to handle the case where after the > last check and before enabling interrupts, an interrupt has been


  • 5.  [PATCH] virtio-ccw: Switch to device/driver language.

    Posted 11-29-2013 13:28
    Switch the parts of the virtio-ccw specification that did not already to device/driver instead of guest/host. For more general discussions, host/guest is kept (like when talking about guest<->host notifications in general). This should make things more consistent. Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com> --- virtio-v1.0-wd01-part1-specification.txt 38 +++++++++++++++--------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/virtio-v1.0-wd01-part1-specification.txt b/virtio-v1.0-wd01-part1-specification.txt index ba3ecae..d7f956d 100644 --- a/virtio-v1.0-wd01-part1-specification.txt +++ b/virtio-v1.0-wd01-part1-specification.txt @@ -1952,7 +1952,7 @@ implies that the device must reject any command not valid for revision 2.3.3.2.2. Configuring a Virtqueue ---------------------------------- -CCW_CMD_READ_VQ_CONF is issued by the guest to obtain information +CCW_CMD_READ_VQ_CONF is issued by the driver to obtain information about a queue. It uses the following structure for communicating: struct vq_config_block { @@ -1963,8 +1963,8 @@ about a queue. It uses the following structure for communicating: The requested number of buffers for queue index is returned in max_num. -Afterwards, CCW_CMD_SET_VQ is issued by the guest to inform the -host about the location used for its queue. The transmitted +Afterwards, CCW_CMD_SET_VQ is issued by the driver to inform the +device about the location used for its queue. The transmitted structure is struct vq_info_block { @@ -2020,7 +2020,7 @@ The calculation for total size is as follows: 2.3.3.2.3. Communicating Status Information ------------------------------------------- -The guest can change the status of a device via the +The driver can change the status of a device via the CCW_CMD_WRITE_STATUS command, which transmits an 8 bit status value. @@ -2043,11 +2043,11 @@ features are the 32 bits of features currently accessed, while index describes which of the feature bit values is to be accessed. -The guest may obtain the host's device feature set via the -CCW_CMD_READ_FEAT command. The host stores the features at index +The guest may obtain the device's device feature set via the +CCW_CMD_READ_FEAT command. The device stores the features at index to features. -For communicating its device features to the host, the guest may +For communicating its supported features to the device, the driver may use the CCW_CMD_WRITE_FEAT command, denoting a features/index combination. @@ -2057,16 +2057,16 @@ combination. The device's configuration space is located in host memory. It is the same size as the standard PCI configuration space. -To obtain information from the configuration space, the guest may -use CCW_CMD_READ_CONF, specifying the guest memory for the host +To obtain information from the configuration space, the driver may +use CCW_CMD_READ_CONF, specifying the guest memory for the device to write to. -For changing configuration information, the guest may use -CCW_CMD_WRITE_CONF, specifying the guest memory for the host to +For changing configuration information, the driver may use +CCW_CMD_WRITE_CONF, specifying the guest memory for the device to read from. In both cases, the complete configuration space is transmitted. This -allows the guest to compare the new configuration space with the old +allows the driver to compare the new configuration space with the old version, and keep a generation count internally whenever it changes. 2.3.3.2.6. Setting Up Indicators @@ -2090,7 +2090,7 @@ Indicators for notification via classic I/O interrupts are contained in a 64 bit value per virtio-ccw proxy device. To communicate the location of the indicator bits for host->guest -notification, the guest uses the CCW_CMD_SET_IND command, +notification, the driver uses the CCW_CMD_SET_IND command, pointing to a location containing the guest address of the indicators in a 64 bit value. @@ -2170,12 +2170,12 @@ If the driver used the CCW_CMD_SET_IND command to set up queue indicators, the device will use classic I/O interrupts for host->guest notification about virtqueue activity. -For notifying the guest of virtqueue buffers, the host sets the +For notifying the driver of virtqueue buffers, the device sets the corresponding bit in the guest-provided indicators. If an -interrupt is not already pending for the subchannel, the host +interrupt is not already pending for the subchannel, the device generates an unsolicited I/O interrupt. -If the host wants to notify the guest about configuration +If the device wants to notify the driver about configuration changes, it sets bit 0 in the configuration indicators and generates an unsolicited I/O interrupt, if needed. This also applies if adapter I/O interrupts are used for queue notifications. @@ -2187,7 +2187,7 @@ If the driver used the CCW_CMD_SET_IND_ADAPTER command to set up queue indicators, the device will use adapter I/O interrupts for host->guest notification about virtqueue activity. -For notifying the guest of virtqueue buffers, the host sets the +For notifying the driver of virtqueue buffers, the device sets the bit in the guest-provided indicator area at the corresponding offset. The guest-provided summary indicator is also set. An adapter I/O interrupt for the corresponding interruption subclass is generated. @@ -2205,7 +2205,7 @@ host->guest notification will always be done via classic I/O interrupts. 2.3.3.3.2. Guest->Host Notification ----------------------------------- -For notifying the host of virtqueue buffers, the guest +For notifying the device of virtqueue buffers, the driver unfortunately can't use a channel command (the asynchronous characteristics of channel I/O interact badly with the host block I/O backend). Instead, it uses a diagnose 0x500 call with subcode @@ -2242,7 +2242,7 @@ used. 2.3.3.3.4. Resetting Devices ---------------------------- -In order to reset a device, a guest may send the +In order to reset a device, a driver may send the CCW_CMD_VDEV_RESET command. -- 1.7.9.5


  • 6.  Re: [PATCH 1/3] Terminology: Device and driver, not host and guest.

    Posted 12-02-2013 01:05
    Cornelia Huck <cornelia.huck@de.ibm.com> writes: > On Tue, 26 Nov 2013 13:41:13 +1030 > Rusty Russell <rusty@au1.ibm.com> wrote: > >> We've mixed both together, whereas from a spec-reader point of view, >> "driver" and "device" is probably clearer. >> >> I avoided changing the CCW part of the spec, since the guest/host >> language is probably more typical in that world. > > OTOH, having different terminology for different transports might be > confusing (and we do talk about devices presented via a control unit). > The uncommon term is "driver"; s390 documentation usually talks about > "the program". host/guest makes sense when talking about the whole > picture as in host->guest notification. > > I'll talk to some folks here to get a second opinion on this. Thanks for fixes, I've applied them to the tex version (along with my previous changes) and am about to commit to SVN once it's all done. Cheers, Rusty. >> >> Signed-off-by: Rusty Russell <rusty@au.ibm.com> >> --- >> virtio-v1.0-wd01-part1-specification.txt 255 ++++++++++++++++--------------- >> 1 file changed, 134 insertions(+), 121 deletions(-) >> >> diff --git a/virtio-v1.0-wd01-part1-specification.txt b/virtio-v1.0-wd01-part1-specification.txt >> index 3be1bef..3ca180f 100644 >> --- a/virtio-v1.0-wd01-part1-specification.txt >> +++ b/virtio-v1.0-wd01-part1-specification.txt > >> @@ -98,7 +111,7 @@ To reinforce this the examples use typenames like "le16" instead of "uint16_t". >> 2.1.1. Device Status Field >> ------------------------- >> >> -The Device Status field is updated by the guest to indicate its >> +The Device Status field is updated by the OS and driver to indicate its > > How about "guest/driver"? We haven't defined OS anywhere :) > >> progress. This provides a simple low-level diagnostic: it's most >> useful to imagine them hooked up to traffic lights on the console >> indicating the status of each device. >> @@ -119,7 +132,7 @@ This field is 0 upon reset, otherwise at least one bit should be set: >> DRIVER_OK (4) Indicates that the driver is set up and ready to >> drive the device. >> >> - FAILED (128) Indicates that something went wrong in the guest, >> + FAILED (128) Indicates that something went wrong in the guest OS, > > I'd keep this at "guest". > >> and it has given up on the device. This could be an internal >> error, or the driver didn't like the device for some reason, or >> even a fatal error during device operation. The device must be > >> @@ -757,11 +770,11 @@ buffer: >> at 65536 as well: >> (u16)(new_idx - used_event - 1) < (u16)(new_idx - old_idx) >> >> -For each ring, guest should then disable interrupts by writing >> +For each ring, driver should then disable interrupts by writing > > s/driver/the driver/ > >> VRING_AVAIL_F_NO_INTERRUPT flag in avail structure, if required. >> It can then process used ring entries finally enabling interrupts >> by clearing the VRING_AVAIL_F_NO_INTERRUPT flag or updating the >> -EVENT_IDX field in the available structure. The guest should then >> +EVENT_IDX field in the available structure. The driver should then >> execute a memory barrier, and then recheck the ring empty >> condition. This is necessary to handle the case where after the >> last check and before enabling interrupts, an interrupt has been