virtio-comment

 View Only
  • 1.  [PATCH v3] virtio-net: Define per-packet hash reporting feature

    Posted 02-05-2020 18:59
    Fixes: https://github.com/oasis-tcs/virtio-spec/issues/66
    Define respective feature bit for virtio-net.
    Extend packet layout to populate hash value and type.
    Move the definition of IP/TCP/UDP header fields to
    calculate the hash out of RSS section to common network
    device section.

    Signed-off-by: Yuri Benditovich <yuri.benditovich@daynix.com>
    ---
    content.tex | 347 ++++++++++++++++++++++++++++++++++------------------
    1 file changed, 228 insertions(+), 119 deletions(-)

    diff --git a/content.tex b/content.tex
    index fb10061..20d598d 100644
    --- a/content.tex
    +++ b/content.tex
    @@ -2895,6 +2895,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
    \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
    channel.

    +\item[VIRTIO_NET_F_HASH_REPORT(58)] Device can report per-packet hash
    + value and a type of calculated hash.
    +
    \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
    value. Device benefits from knowing the exact header length.

    @@ -2981,35 +2984,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
    le32 supported_hash_types;
    };
    \end{lstlisting}
    -\label{sec:Device Types / Network Device / Device configuration layout / RSS}
    -Three following fields, \field{rss_max_key_size}, \field{rss_max_indirection_table_length}
    -and \field{supported_hash_types} only exist if VIRTIO_NET_F_RSS is set.
    +The following field, \field{rss_max_key_size} only exist if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
    +It specifies the maximal supported length of RSS key in bytes.

    -Field \field{rss_max_key_size} specifies the maximal supported length of RSS key in bytes.
    +The following field, \field{rss_max_indirection_table_length} only exist if VIRTIO_NET_F_RSS is set.
    +It specifies the maximal number of 16-bit entries in RSS indirection table.

    -Field \field{rss_max_indirection_table_length} specifies the maximal number of 16-bit entries in RSS indirection table.
    +The next field, \field{supported_hash_types} exists only if the device supports hash calculation,
    +i.e. if VIRTIO_NET_F_RSS of VIRTIO_NET_F_HASH_REPORT is set.

    -Field \field{supported_hash_types} contains the bitmask of supported RSS hash types.
    -
    -Hash types applicable for IPv4 packets:
    -\begin{lstlisting}
    -#define VIRTIO_NET_RSS_HASH_TYPE_IPv4 (1 << 0)
    -#define VIRTIO_NET_RSS_HASH_TYPE_TCPv4 (1 << 1)
    -#define VIRTIO_NET_RSS_HASH_TYPE_UDPv4 (1 << 2)
    -\end{lstlisting}
    -Hash types applicable for IPv6 packets without extension headers
    -\begin{lstlisting}
    -#define VIRTIO_NET_RSS_HASH_TYPE_IPv6 (1 << 3)
    -#define VIRTIO_NET_RSS_HASH_TYPE_TCPv6 (1 << 4)
    -#define VIRTIO_NET_RSS_HASH_TYPE_UDPv6 (1 << 5)
    -\end{lstlisting}
    -Hash types applicable for IPv6 packets with extension headers
    -\begin{lstlisting}
    -#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX (1 << 6)
    -#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX (1 << 7)
    -#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX (1 << 8)
    -\end{lstlisting}
    -For the exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
    +Field \field{supported_hash_types} contains the bitmask of supported hash types.
    +See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} for details of supported hash types.

    \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}

    @@ -3034,7 +3019,7 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
    negotiated.

    The device MUST set \field{rss_max_key_size} to at least 40, if it offers
    -VIRTIO_NET_F_RSS.
    +VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.

    The device MUST set \field{rss_max_indirection_table_length} to at least 128, if it offers
    VIRTIO_NET_F_RSS.
    @@ -3159,6 +3144,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
    le16 csum_start;
    le16 csum_offset;
    le16 num_buffers;
    + le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
    + le16 hash_type; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
    };
    \end{lstlisting}

    @@ -3481,6 +3468,12 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network

    \end{enumerate}

    +If applicable, the device calculates per-packet hash for incoming packets as
    +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets}.
    +
    +If applicable, the device reports hash information for incoming packets as
    +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}.
    +
    \devicenormative{\paragraph}{Processing of Incoming Packets}{Device Types / Network Device / Device Operation / Processing of Incoming Packets}
    \label{devicenormative:Device Types / Network Device / Device Operation / Processing of Packets}%old label for latexdiff

    @@ -3583,6 +3576,181 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
    If neither VIRTIO_NET_HDR_F_NEEDS_CSUM nor
    VIRTIO_NET_HDR_F_DATA_VALID is set, the driver MUST NOT
    rely on the packet checksum being correct.
    +
    +\paragraph{Hash calculation for incoming packets}
    +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets}
    +
    +A device attempts to calculate a per-packet hash in the following cases:
    +\begin{itemize}
    +\item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
    +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
    +\end{itemize}
    +
    +If the feature VIRTIO_NET_F_RSS was negotiated:
    +\begin{itemize}
    +\item The device uses \field{hash_types} of the virtio_net_rss_config structure as the 'Enabled hash types' bitmask.
    +\item The device uses \field{hash_key_data} of the virtio_net_rss_config structure.
    +\end{itemize}
    +See the definition of virtio_net_rss_config structure in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
    +
    +If the feature VIRTIO_NET_F_RSS was not negotiated:
    +\begin{itemize}
    +\item The device uses \field{hash_types} of the virtio_net_hash_config structure as the 'Enabled hash types' bitmask.
    +\item The device uses \field{hash_key_data} of the virtio_net_hash_config structure.
    +\end{itemize}
    +See the definition of virtio_net_hash_config structure in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
    +
    +Note that if the device offers VIRTIO_NET_F_HASH_REPORT, even if it supports only one pair of virtqueues, it SHOULD support
    +at least one of commands of VIRTIO_NET_CTRL_MQ class to configure reported hash parameters:
    +\begin{itemize}
    +\item If the device offers VIRTIO_NET_F_RSS, it MUST support VIRTIO_NET_CTRL_MQ_RSS_CONFIG command per
    + \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}.
    +\item Otherwise the device SHOULD support VIRTIO_NET_CTRL_MQ_HASH_CONFIG command per
    + \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
    +\end{itemize}
    +
    +\subparagraph{Supported/enabled hash types}
    +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
    +Hash types applicable for IPv4 packets:
    +\begin{lstlisting}
    +#define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
    +#define VIRTIO_NET_HASH_TYPE_TCPv4 (1 << 1)
    +#define VIRTIO_NET_HASH_TYPE_UDPv4 (1 << 2)
    +\end{lstlisting}
    +Hash types applicable for IPv6 packets without extension headers
    +\begin{lstlisting}
    +#define VIRTIO_NET_HASH_TYPE_IPv6 (1 << 3)
    +#define VIRTIO_NET_HASH_TYPE_TCPv6 (1 << 4)
    +#define VIRTIO_NET_HASH_TYPE_UDPv6 (1 << 5)
    +\end{lstlisting}
    +Hash types applicable for IPv6 packets with extension headers
    +\begin{lstlisting}
    +#define VIRTIO_NET_HASH_TYPE_IP_EX (1 << 6)
    +#define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
    +#define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
    +\end{lstlisting}
    +
    +\subparagraph{IPv4 packets}
    +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
    +The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
    +\begin{itemize}
    +\item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the packet has
    +a TCP header, the hash is calculated over the following fields:
    +\begin{itemize}
    +\item Source IP address
    +\item Destination IP address
    +\item Source TCP port
    +\item Destination TCP port
    +\end{itemize}
    +\item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the
    +packet has a UDP header, the hash is calculated over the following fields:
    +\begin{itemize}
    +\item Source IP address
    +\item Destination IP address
    +\item Source UDP port
    +\item Destination UDP port
    +\end{itemize}
    +\item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is
    +calculated over the following fields:
    +\begin{itemize}
    +\item Source IP address
    +\item Destination IP address
    +\end{itemize}
    +\item Else the device does not calculate the hash
    +\end{itemize}
    +
    +\subparagraph{IPv6 packets without extension header}
    +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}
    +The device calculates the hash on IPv6 packets without extension
    +headers according to 'Enabled hash types' bitmask as follows:
    +\begin{itemize}
    +\item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the packet has
    +a TCPv6 header, the hash is calculated over the following fields:
    +\begin{itemize}
    +\item Source IPv6 address
    +\item Destination IPv6 address
    +\item Source TCP port
    +\item Destination TCP port
    +\end{itemize}
    +\item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the
    +packet has a UDPv6 header, the hash is calculated over the following fields:
    +\begin{itemize}
    +\item Source IPv6 address
    +\item Destination IPv6 address
    +\item Source UDP port
    +\item Destination UDP port
    +\end{itemize}
    +\item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is
    +calculated over the following fields:
    +\begin{itemize}
    +\item Source IPv6 address
    +\item Destination IPv6 address
    +\end{itemize}
    +\item Else the device does not calculate the hash
    +\end{itemize}
    +
    +\subparagraph{IPv6 packets with extension header}
    +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets with extension header}
    +The device calculates the hash on IPv6 packets with extension
    +headers according to 'Enabled hash types' bitmask as follows:
    +\begin{itemize}
    +\item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the packet
    +has a TCPv6 header, the hash is calculated over the following fields:
    +\begin{itemize}
    +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    +\item Source TCP port
    +\item Destination TCP port
    +\end{itemize}
    +\item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the
    +packet has a UDPv6 header, the hash is calculated over the following fields:
    +\begin{itemize}
    +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    +\item Source UDP port
    +\item Destination UDP port
    +\end{itemize}
    +\item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is
    +calculated over the following fields:
    +\begin{itemize}
    +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    +\end{itemize}
    +\item Else skip IPv6 extension headers and calculate the hash as
    +defined for an IPv6 packet without extension headers
    +(see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
    +\end{itemize}
    +
    +\paragraph{Hash reporting for incoming packets}
    +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}%old label for latexdiff
    +
    +If feature VIRTIO_NET_F_HASH_REPORT was negotiated and
    + the device has calculated the hash for the packet, the device fills \field{hash_type} with the type of calculated hash
    +and \field{hash_value} with the value of calculated hash.
    +
    +If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
    +hash was not calculated, the device sets \field{hash_type} to VIRTIO_NET_HASH_REPORT_NONE.
    +
    +Possible values that the device can report in \field{hash_type} are defined below.
    +They correspond to supported hash types defined in
    +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
    +as follows:
    +
    +VIRTIO_NET_HASH_TYPE_XXX = 1 << (VIRTIO_NET_HASH_REPORT_XXX - 1)
    +
    +\begin{lstlisting}
    +#define VIRTIO_NET_HASH_REPORT_NONE 0
    +#define VIRTIO_NET_HASH_REPORT_IPv4 1
    +#define VIRTIO_NET_HASH_REPORT_TCPv4 2
    +#define VIRTIO_NET_HASH_REPORT_UDPv4 3
    +#define VIRTIO_NET_HASH_REPORT_IPv6 4
    +#define VIRTIO_NET_HASH_REPORT_TCPv6 5
    +#define VIRTIO_NET_HASH_REPORT_UDPv6 6
    +#define VIRTIO_NET_HASH_REPORT_IPv6_EX 7
    +#define VIRTIO_NET_HASH_REPORT_TCPv6_EX 8
    +#define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
    +\end{lstlisting}
    +
    \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}

    The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
    @@ -3860,6 +4028,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    #define VIRTIO_NET_CTRL_MQ 4
    #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET 0 (for automatic receive steering)
    #define VIRTIO_NET_CTRL_MQ_RSS_CONFIG 1 (for configurable receive steering)
    + #define VIRTIO_NET_CTRL_MQ_HASH_CONFIG 2 (for configurable hash calculation)
    \end{lstlisting}

    If more than one multiqueue mode is negotiated, the resulting device configuration is defined by the last command sent by the driver.
    @@ -3925,10 +4094,37 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    according to the native endian of the guest rather than
    (necessarily when not using the legacy interface) little-endian.

    +\subparagraph{Hash calculation}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}
    +If VIRTIO_NET_F_HASH_REPORT negotiated and the device uses automatic receive steering,
    +the device SHOULD support a command to configure hash calculation parameters.
    +
    +The driver provides parameters for hash calculation as follows:
    +
    +\field{class} VIRTIO_NET_CTRL_MQ, \field{command} VIRTIO_NET_CTRL_MQ_HASH_CONFIG.
    +
    +The \field{command-specific-data} has following format:
    +\begin{lstlisting}
    +struct virtio_net_hash_config {
    + le32 hash_types;
    + le16 reserved[4];
    + u8 hash_key_length;
    + u8 hash_key_data[hash_key_length];
    +};
    +\end{lstlisting}
    +Field \field{hash_types} contains a bitmask of allowed hash types as
    +defined in
    +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
    +Initially the device has all hash types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
    +
    +Field \field{reserved} MUST contain zeroes. It is defined to make the structure to look similar to virtio_net_rss_config structure,
    +defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}.
    +
    +Fields \field{hash_key_length} and \field{hash_key_data} define the key to be used in hash calculation.
    +
    \paragraph{Receive-side scaling (RSS)}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}
    A device offers the feature VIRTIO_NET_F_RSS if it supports RSS receive steering with Toeplitz hash calculation and configurable parameters.

    -A driver queries RSS capabilities of the device by reading device configuration as defined in \ref{sec:Device Types / Network Device / Device configuration layout / RSS}
    +A driver queries RSS capabilities of the device by reading device configuration as defined in \ref{sec:Device Types / Network Device / Device configuration layout}

    \subparagraph{Setting RSS parameters}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}

    @@ -3945,7 +4141,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    };
    \end{lstlisting}
    Field \field{hash_types} contains a bitmask of allowed hash types as
    -defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
    +defined in
    +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.

    Field \field{indirection_table_mask} is a mask to be applied to
    the calculated hash to produce an index in the
    @@ -3959,94 +4156,6 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi

    A driver sets \field{max_tx_vq} to inform a device how many transmit virtqueues it may use (transmitq1\ldots transmitq \field{max_tx_vq}).

    -\subparagraph{RSS hash types}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
    -
    -The device calculates the hash on IPv4 packets according to the
    -field \field{hash_types} of the virtio_net_rss_config structure as follows:
    -\begin{itemize}
    -\item If VIRTIO_NET_RSS_HASH_TYPE_TCPv4 is set and the packet has
    -a TCP header, the hash is calculated over the following fields:
    -\begin{itemize}
    -\item Source IP address
    -\item Destination IP address
    -\item Source TCP port
    -\item Destination TCP port
    -\end{itemize}
    -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDPv4 is set and the
    -packet has a UDP header, the hash is calculated over the following fields:
    -\begin{itemize}
    -\item Source IP address
    -\item Destination IP address
    -\item Source UDP port
    -\item Destination UDP port
    -\end{itemize}
    -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IPv4 is set, the hash is
    -calculated over the following fields:
    -\begin{itemize}
    -\item Source IP address
    -\item Destination IP address
    -\end{itemize}
    -\item Else the device does not calculate the hash
    -\end{itemize}
    -
    -\label{itm:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types/hash on IPv6 packets without extension headers}
    -The device calculates the hash on IPv6 packets without extension
    -headers according to the field \field{hash_types} of the virtio_net_rss_config structure as follows:
    -\begin{itemize}
    -\item If VIRTIO_NET_RSS_HASH_TYPE_TCPv6 is set and the packet has
    -a TCPv6 header, the hash is calculated over the following fields:
    -\begin{itemize}
    -\item Source IPv6 address
    -\item Destination IPv6 address
    -\item Source TCP port
    -\item Destination TCP port
    -\end{itemize}
    -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDPv6 is set and the
    -packet has a UDPv6 header, the hash is calculated over the following fields:
    -\begin{itemize}
    -\item Source IPv6 address
    -\item Destination IPv6 address
    -\item Source UDP port
    -\item Destination UDP port
    -\end{itemize}
    -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IPv6 is set, the hash is
    -calculated over the following fields:
    -\begin{itemize}
    -\item Source IPv6 address
    -\item Destination IPv6 address
    -\end{itemize}
    -\item Else the device does not calculate the hash
    -\end{itemize}
    -
    -The device calculates the hash on IPv6 packets with extension
    -headers according to the field \field{hash_types} of the virtio_net_rss_config structure as follows:
    -\begin{itemize}
    -\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet
    -has a TCPv6 header, the hash is calculated over the following fields:
    -\begin{itemize}
    -\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    -\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    -\item Source TCP port
    -\item Destination TCP port
    -\end{itemize}
    -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the
    -packet has a UDPv6 header, the hash is calculated over the following fields:
    -\begin{itemize}
    -\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    -\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    -\item Source UDP port
    -\item Destination UDP port
    -\end{itemize}
    -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is
    -calculated over the following fields:
    -\begin{itemize}
    -\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    -\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    -\end{itemize}
    -\item Else skip IPv6 extension headers and calculate the hash as
    -defined for an IPv6 packet without extension headers (see \ref{itm:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types/hash on IPv6 packets without extension headers}).
    -\end{itemize}
    -
    \drivernormative{\subparagraph}{Setting RSS parameters}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) }

    A driver MUST NOT send the VIRTIO_NET_CTRL_MQ_RSS_CONFIG command if the feature VIRTIO_NET_F_RSS has not been negotiated.
    @@ -4057,12 +4166,12 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi

    A driver MUST use \field{indirection_table_mask} values that are less than \field{rss_max_indirection_table_length} reported by a device.

    -A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not supported by a device.
    +A driver MUST NOT set any VIRTIO_NET_HASH_TYPE_ flags that are not supported by a device.

    \devicenormative{\subparagraph}{RSS processing}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
    The device MUST determine the destination queue for a network packet as follows:
    \begin{itemize}
    -\item Calculate the hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
    +\item Calculate the hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets}.
    \item If the device did not calculate the hash for the specific packet, the device directs the packet to the receiveq specified by \field{unclassified_queue} of virtio_net_rss_config structure (value of 0 corresponds to receiveq1).
    \item Apply \field{indirection_table_mask} to the calculated hash and use the result as the index in the indirection table to get 0-based number of destination receiveq (value of 0 corresponds to receiveq1).
    \end{itemize}
    --
    2.17.1




  • 2.  Re: [PATCH v3] virtio-net: Define per-packet hash reporting feature

    Posted 02-18-2020 05:59
    On Wed, Feb 05, 2020 at 08:59:05PM +0200, Yuri Benditovich wrote:
    > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/66
    > Define respective feature bit for virtio-net.
    > Extend packet layout to populate hash value and type.
    > Move the definition of IP/TCP/UDP header fields to
    > calculate the hash out of RSS section to common network
    > device section.
    >
    > Signed-off-by: Yuri Benditovich <yuri.benditovich@daynix.com>

    Looks good to me.
    Minor typos/tweaks.
    Otherwise

    Acked-by: Michael S. Tsirkin <mst@redhat.com>


    > ---
    > content.tex | 347 ++++++++++++++++++++++++++++++++++------------------
    > 1 file changed, 228 insertions(+), 119 deletions(-)
    >
    > diff --git a/content.tex b/content.tex
    > index fb10061..20d598d 100644
    > --- a/content.tex
    > +++ b/content.tex
    > @@ -2895,6 +2895,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
    > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
    > channel.
    >
    > +\item[VIRTIO_NET_F_HASH_REPORT(58)] Device can report per-packet hash
    > + value and a type of calculated hash.
    > +
    > \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
    > value. Device benefits from knowing the exact header length.
    >
    > @@ -2981,35 +2984,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
    > le32 supported_hash_types;
    > };
    > \end{lstlisting}
    > -\label{sec:Device Types / Network Device / Device configuration layout / RSS}
    > -Three following fields, \field{rss_max_key_size}, \field{rss_max_indirection_table_length}
    > -and \field{supported_hash_types} only exist if VIRTIO_NET_F_RSS is set.
    > +The following field, \field{rss_max_key_size} only exist if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
    > +It specifies the maximal supported length of RSS key in bytes.
    >
    > -Field \field{rss_max_key_size} specifies the maximal supported length of RSS key in bytes.
    > +The following field, \field{rss_max_indirection_table_length} only exist if VIRTIO_NET_F_RSS is set.
    > +It specifies the maximal number of 16-bit entries in RSS indirection table.
    >
    > -Field \field{rss_max_indirection_table_length} specifies the maximal number of 16-bit entries in RSS indirection table.
    > +The next field, \field{supported_hash_types} exists only if the device supports hash calculation,
    > +i.e. if VIRTIO_NET_F_RSS of VIRTIO_NET_F_HASH_REPORT is set.

    of -> or?

    >
    > -Field \field{supported_hash_types} contains the bitmask of supported RSS hash types.
    > -
    > -Hash types applicable for IPv4 packets:
    > -\begin{lstlisting}
    > -#define VIRTIO_NET_RSS_HASH_TYPE_IPv4 (1 << 0)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_TCPv4 (1 << 1)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_UDPv4 (1 << 2)
    > -\end{lstlisting}
    > -Hash types applicable for IPv6 packets without extension headers
    > -\begin{lstlisting}
    > -#define VIRTIO_NET_RSS_HASH_TYPE_IPv6 (1 << 3)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_TCPv6 (1 << 4)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_UDPv6 (1 << 5)
    > -\end{lstlisting}
    > -Hash types applicable for IPv6 packets with extension headers
    > -\begin{lstlisting}
    > -#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX (1 << 6)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX (1 << 7)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX (1 << 8)
    > -\end{lstlisting}
    > -For the exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
    > +Field \field{supported_hash_types} contains the bitmask of supported hash types.
    > +See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} for details of supported hash types.
    >
    > \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
    >
    > @@ -3034,7 +3019,7 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
    > negotiated.
    >
    > The device MUST set \field{rss_max_key_size} to at least 40, if it offers
    > -VIRTIO_NET_F_RSS.
    > +VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
    >
    > The device MUST set \field{rss_max_indirection_table_length} to at least 128, if it offers
    > VIRTIO_NET_F_RSS.
    > @@ -3159,6 +3144,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
    > le16 csum_start;
    > le16 csum_offset;
    > le16 num_buffers;
    > + le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
    > + le16 hash_type; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
    > };
    > \end{lstlisting}
    >
    > @@ -3481,6 +3468,12 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
    >
    > \end{enumerate}
    >
    > +If applicable, the device calculates per-packet hash for incoming packets as
    > +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets}.
    > +
    > +If applicable, the device reports hash information for incoming packets as
    > +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}.
    > +
    > \devicenormative{\paragraph}{Processing of Incoming Packets}{Device Types / Network Device / Device Operation / Processing of Incoming Packets}
    > \label{devicenormative:Device Types / Network Device / Device Operation / Processing of Packets}%old label for latexdiff
    >
    > @@ -3583,6 +3576,181 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
    > If neither VIRTIO_NET_HDR_F_NEEDS_CSUM nor
    > VIRTIO_NET_HDR_F_DATA_VALID is set, the driver MUST NOT
    > rely on the packet checksum being correct.
    > +
    > +\paragraph{Hash calculation for incoming packets}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets}
    > +
    > +A device attempts to calculate a per-packet hash in the following cases:
    > +\begin{itemize}
    > +\item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
    > +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
    > +\end{itemize}

    below mentions structures that were never mentioned earlier.
    We should say a couple of words here.
    E.g.
    the driver configures the hash calculation by sending
    either the virtio_net_rss_config structure as described in
    ... or the
    virtio_net_hash_config structure as described in ... to the device.


    > +If the feature VIRTIO_NET_F_RSS was negotiated:
    > +\begin{itemize}
    > +\item The device uses \field{hash_types} of the virtio_net_rss_config structure as the 'Enabled hash types' bitmask.
    > +\item The device uses \field{hash_key_data} of the virtio_net_rss_config structure.

    .. as ... ?

    Also what about key length field?
    We should document all fields.

    Also I would maybe repeat this near where the structure itself is defined.

    > +\end{itemize}
    > +See the definition of virtio_net_rss_config structure in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}

    I would just add a link where virtio_net_rss_config is mentioned.
    E.g.
    The device uses \field{hash_key_data} of the virtio_net_rss_config
    structure (See \ref{...}).


    > +
    > +If the feature VIRTIO_NET_F_RSS was not negotiated:
    > +\begin{itemize}
    > +\item The device uses \field{hash_types} of the virtio_net_hash_config structure as the 'Enabled hash types' bitmask.
    > +\item The device uses \field{hash_key_data} of the virtio_net_hash_config structure.
    > +\end{itemize}
    > +See the definition of virtio_net_hash_config structure in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
    > +
    > +Note that if the device offers VIRTIO_NET_F_HASH_REPORT, even if it supports only one pair of virtqueues, it SHOULD support
    > +at least one of commands of VIRTIO_NET_CTRL_MQ class to configure reported hash parameters:
    > +\begin{itemize}
    > +\item If the device offers VIRTIO_NET_F_RSS, it MUST support VIRTIO_NET_CTRL_MQ_RSS_CONFIG command per
    > + \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}.
    > +\item Otherwise the device SHOULD support VIRTIO_NET_CTRL_MQ_HASH_CONFIG command per
    > + \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
    > +\end{itemize}
    > +
    > +\subparagraph{Supported/enabled hash types}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
    > +Hash types applicable for IPv4 packets:
    > +\begin{lstlisting}
    > +#define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
    > +#define VIRTIO_NET_HASH_TYPE_TCPv4 (1 << 1)
    > +#define VIRTIO_NET_HASH_TYPE_UDPv4 (1 << 2)
    > +\end{lstlisting}
    > +Hash types applicable for IPv6 packets without extension headers
    > +\begin{lstlisting}
    > +#define VIRTIO_NET_HASH_TYPE_IPv6 (1 << 3)
    > +#define VIRTIO_NET_HASH_TYPE_TCPv6 (1 << 4)
    > +#define VIRTIO_NET_HASH_TYPE_UDPv6 (1 << 5)
    > +\end{lstlisting}
    > +Hash types applicable for IPv6 packets with extension headers
    > +\begin{lstlisting}
    > +#define VIRTIO_NET_HASH_TYPE_IP_EX (1 << 6)
    > +#define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
    > +#define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
    > +\end{lstlisting}
    > +
    > +\subparagraph{IPv4 packets}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
    > +The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
    > +\begin{itemize}
    > +\item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the packet has
    > +a TCP header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IP address
    > +\item Destination IP address
    > +\item Source TCP port
    > +\item Destination TCP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the
    > +packet has a UDP header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IP address
    > +\item Destination IP address
    > +\item Source UDP port
    > +\item Destination UDP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is
    > +calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IP address
    > +\item Destination IP address
    > +\end{itemize}
    > +\item Else the device does not calculate the hash
    > +\end{itemize}
    > +
    > +\subparagraph{IPv6 packets without extension header}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}
    > +The device calculates the hash on IPv6 packets without extension
    > +headers according to 'Enabled hash types' bitmask as follows:
    > +\begin{itemize}
    > +\item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the packet has
    > +a TCPv6 header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IPv6 address
    > +\item Destination IPv6 address
    > +\item Source TCP port
    > +\item Destination TCP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the
    > +packet has a UDPv6 header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IPv6 address
    > +\item Destination IPv6 address
    > +\item Source UDP port
    > +\item Destination UDP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is
    > +calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IPv6 address
    > +\item Destination IPv6 address
    > +\end{itemize}
    > +\item Else the device does not calculate the hash
    > +\end{itemize}
    > +
    > +\subparagraph{IPv6 packets with extension header}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets with extension header}
    > +The device calculates the hash on IPv6 packets with extension
    > +headers according to 'Enabled hash types' bitmask as follows:
    > +\begin{itemize}
    > +\item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the packet
    > +has a TCPv6 header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > +\item Source TCP port
    > +\item Destination TCP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the
    > +packet has a UDPv6 header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > +\item Source UDP port
    > +\item Destination UDP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is
    > +calculated over the following fields:
    > +\begin{itemize}
    > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > +\end{itemize}
    > +\item Else skip IPv6 extension headers and calculate the hash as
    > +defined for an IPv6 packet without extension headers
    > +(see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
    > +\end{itemize}
    > +
    > +\paragraph{Hash reporting for incoming packets}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}%old label for latexdiff

    I doubt it's an old label

    > +
    > +If feature VIRTIO_NET_F_HASH_REPORT

    If feature VIRTIO_NET_F_HASH_REPORT -> If VIRTIO_NET_F_HASH_REPORT

    > was negotiated and
    > + the device has calculated the hash for the packet, the device fills \field{hash_type} with the type of calculated hash
    > +and \field{hash_value} with the value of calculated hash.
    > +
    > +If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
    > +hash was not calculated, the device sets \field{hash_type} to VIRTIO_NET_HASH_REPORT_NONE.
    > +
    > +Possible values that the device can report in \field{hash_type} are defined below.
    > +They correspond to supported hash types defined in
    > +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
    > +as follows:
    > +
    > +VIRTIO_NET_HASH_TYPE_XXX = 1 << (VIRTIO_NET_HASH_REPORT_XXX - 1)
    > +
    > +\begin{lstlisting}
    > +#define VIRTIO_NET_HASH_REPORT_NONE 0
    > +#define VIRTIO_NET_HASH_REPORT_IPv4 1
    > +#define VIRTIO_NET_HASH_REPORT_TCPv4 2
    > +#define VIRTIO_NET_HASH_REPORT_UDPv4 3
    > +#define VIRTIO_NET_HASH_REPORT_IPv6 4
    > +#define VIRTIO_NET_HASH_REPORT_TCPv6 5
    > +#define VIRTIO_NET_HASH_REPORT_UDPv6 6
    > +#define VIRTIO_NET_HASH_REPORT_IPv6_EX 7
    > +#define VIRTIO_NET_HASH_REPORT_TCPv6_EX 8
    > +#define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
    > +\end{lstlisting}
    > +
    > \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
    >
    > The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
    > @@ -3860,6 +4028,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    > #define VIRTIO_NET_CTRL_MQ 4
    > #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET 0 (for automatic receive steering)
    > #define VIRTIO_NET_CTRL_MQ_RSS_CONFIG 1 (for configurable receive steering)
    > + #define VIRTIO_NET_CTRL_MQ_HASH_CONFIG 2 (for configurable hash calculation)
    > \end{lstlisting}
    >
    > If more than one multiqueue mode is negotiated, the resulting device configuration is defined by the last command sent by the driver.
    > @@ -3925,10 +4094,37 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    > according to the native endian of the guest rather than
    > (necessarily when not using the legacy interface) little-endian.
    >
    > +\subparagraph{Hash calculation}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}
    > +If VIRTIO_NET_F_HASH_REPORT negotiated

    negotiated -> was negotiated

    > and the device uses automatic receive steering,
    > +the device SHOULD support a command to configure hash calculation parameters.

    SHOULD and not MUST do drivers must expect failure and handle
    gracefully?

    > +
    > +The driver provides parameters for hash calculation as follows:
    > +
    > +\field{class} VIRTIO_NET_CTRL_MQ, \field{command} VIRTIO_NET_CTRL_MQ_HASH_CONFIG.
    > +
    > +The \field{command-specific-data} has following format:
    > +\begin{lstlisting}
    > +struct virtio_net_hash_config {
    > + le32 hash_types;
    > + le16 reserved[4];
    > + u8 hash_key_length;
    > + u8 hash_key_data[hash_key_length];
    > +};
    > +\end{lstlisting}
    > +Field \field{hash_types} contains a bitmask of allowed hash types as
    > +defined in
    > +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
    > +Initially the device has all hash types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
    > +
    > +Field \field{reserved} MUST contain zeroes. It is defined to make the structure to look similar

    look similar -> match the layout of

    > to virtio_net_rss_config structure,
    > +defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}.
    > +
    > +Fields \field{hash_key_length} and \field{hash_key_data} define the key to be used in hash calculation.
    > +
    > \paragraph{Receive-side scaling (RSS)}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}
    > A device offers the feature VIRTIO_NET_F_RSS if it supports RSS receive steering with Toeplitz hash calculation and configurable parameters.
    >
    > -A driver queries RSS capabilities of the device by reading device configuration as defined in \ref{sec:Device Types / Network Device / Device configuration layout / RSS}
    > +A driver queries RSS capabilities of the device by reading device configuration as defined in \ref{sec:Device Types / Network Device / Device configuration layout}
    >
    > \subparagraph{Setting RSS parameters}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
    >
    > @@ -3945,7 +4141,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    > };
    > \end{lstlisting}
    > Field \field{hash_types} contains a bitmask of allowed hash types as
    > -defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
    > +defined in
    > +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
    >
    > Field \field{indirection_table_mask} is a mask to be applied to
    > the calculated hash to produce an index in the
    > @@ -3959,94 +4156,6 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    >
    > A driver sets \field{max_tx_vq} to inform a device how many transmit virtqueues it may use (transmitq1\ldots transmitq \field{max_tx_vq}).
    >
    > -\subparagraph{RSS hash types}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
    > -
    > -The device calculates the hash on IPv4 packets according to the
    > -field \field{hash_types} of the virtio_net_rss_config structure as follows:
    > -\begin{itemize}
    > -\item If VIRTIO_NET_RSS_HASH_TYPE_TCPv4 is set and the packet has
    > -a TCP header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IP address
    > -\item Destination IP address
    > -\item Source TCP port
    > -\item Destination TCP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDPv4 is set and the
    > -packet has a UDP header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IP address
    > -\item Destination IP address
    > -\item Source UDP port
    > -\item Destination UDP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IPv4 is set, the hash is
    > -calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IP address
    > -\item Destination IP address
    > -\end{itemize}
    > -\item Else the device does not calculate the hash
    > -\end{itemize}
    > -
    > -\label{itm:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types/hash on IPv6 packets without extension headers}
    > -The device calculates the hash on IPv6 packets without extension
    > -headers according to the field \field{hash_types} of the virtio_net_rss_config structure as follows:
    > -\begin{itemize}
    > -\item If VIRTIO_NET_RSS_HASH_TYPE_TCPv6 is set and the packet has
    > -a TCPv6 header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IPv6 address
    > -\item Destination IPv6 address
    > -\item Source TCP port
    > -\item Destination TCP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDPv6 is set and the
    > -packet has a UDPv6 header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IPv6 address
    > -\item Destination IPv6 address
    > -\item Source UDP port
    > -\item Destination UDP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IPv6 is set, the hash is
    > -calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IPv6 address
    > -\item Destination IPv6 address
    > -\end{itemize}
    > -\item Else the device does not calculate the hash
    > -\end{itemize}
    > -
    > -The device calculates the hash on IPv6 packets with extension
    > -headers according to the field \field{hash_types} of the virtio_net_rss_config structure as follows:
    > -\begin{itemize}
    > -\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet
    > -has a TCPv6 header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > -\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > -\item Source TCP port
    > -\item Destination TCP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the
    > -packet has a UDPv6 header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > -\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > -\item Source UDP port
    > -\item Destination UDP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is
    > -calculated over the following fields:
    > -\begin{itemize}
    > -\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > -\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > -\end{itemize}
    > -\item Else skip IPv6 extension headers and calculate the hash as
    > -defined for an IPv6 packet without extension headers (see \ref{itm:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types/hash on IPv6 packets without extension headers}).
    > -\end{itemize}
    > -
    > \drivernormative{\subparagraph}{Setting RSS parameters}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) }
    >
    > A driver MUST NOT send the VIRTIO_NET_CTRL_MQ_RSS_CONFIG command if the feature VIRTIO_NET_F_RSS has not been negotiated.
    > @@ -4057,12 +4166,12 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    >
    > A driver MUST use \field{indirection_table_mask} values that are less than \field{rss_max_indirection_table_length} reported by a device.
    >
    > -A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not supported by a device.
    > +A driver MUST NOT set any VIRTIO_NET_HASH_TYPE_ flags that are not supported by a device.
    >
    > \devicenormative{\subparagraph}{RSS processing}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
    > The device MUST determine the destination queue for a network packet as follows:
    > \begin{itemize}
    > -\item Calculate the hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
    > +\item Calculate the hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets}.
    > \item If the device did not calculate the hash for the specific packet, the device directs the packet to the receiveq specified by \field{unclassified_queue} of virtio_net_rss_config structure (value of 0 corresponds to receiveq1).
    > \item Apply \field{indirection_table_mask} to the calculated hash and use the result as the index in the indirection table to get 0-based number of destination receiveq (value of 0 corresponds to receiveq1).
    > \end{itemize}
    > --
    > 2.17.1




  • 3.  Re: [PATCH v3] virtio-net: Define per-packet hash reporting feature

    Posted 02-18-2020 07:44
    ----- Original Message -----

    > From: "Michael S. Tsirkin" <mst@redhat.com>
    > To: "Yuri Benditovich" <yuri.benditovich@daynix.com>
    > Cc: virtio-comment@lists.oasis-open.org, ybendito@redhat.com
    > Sent: Tuesday, February 18, 2020 7:59:13 AM
    > Subject: Re: [PATCH v3] virtio-net: Define per-packet hash reporting feature

    > On Wed, Feb 05, 2020 at 08:59:05PM +0200, Yuri Benditovich wrote:
    > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/66
    > > Define respective feature bit for virtio-net.
    > > Extend packet layout to populate hash value and type.
    > > Move the definition of IP/TCP/UDP header fields to
    > > calculate the hash out of RSS section to common network
    > > device section.
    > >
    > > Signed-off-by: Yuri Benditovich <yuri.benditovich@daynix.com>

    > Looks good to me.
    > Minor typos/tweaks.
    > Otherwise

    > Acked-by: Michael S. Tsirkin <mst@redhat.com>

    > > ---
    > > content.tex | 347 ++++++++++++++++++++++++++++++++++------------------
    > > 1 file changed, 228 insertions(+), 119 deletions(-)
    > >
    > > diff --git a/content.tex b/content.tex
    > > index fb10061..20d598d 100644
    > > --- a/content.tex
    > > +++ b/content.tex
    > > @@ -2895,6 +2895,9 @@ \subsection{Feature bits}\label{sec:Device Types /
    > > Network Device / Feature bits
    > > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
    > > channel.
    > >
    > > +\item[VIRTIO_NET_F_HASH_REPORT(58)] Device can report per-packet hash
    > > + value and a type of calculated hash.
    > > +
    > > \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact
    > > \field{hdr_len}
    > > value. Device benefits from knowing the exact header length.
    > >
    > > @@ -2981,35 +2984,17 @@ \subsection{Device configuration
    > > layout}\label{sec:Device Types / Network Device
    > > le32 supported_hash_types;
    > > };
    > > \end{lstlisting}
    > > -\label{sec:Device Types / Network Device / Device configuration layout /
    > > RSS}
    > > -Three following fields, \field{rss_max_key_size},
    > > \field{rss_max_indirection_table_length}
    > > -and \field{supported_hash_types} only exist if VIRTIO_NET_F_RSS is set.
    > > +The following field, \field{rss_max_key_size} only exist if
    > > VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
    > > +It specifies the maximal supported length of RSS key in bytes.
    > >
    > > -Field \field{rss_max_key_size} specifies the maximal supported length of
    > > RSS key in bytes.
    > > +The following field, \field{rss_max_indirection_table_length} only exist
    > > if VIRTIO_NET_F_RSS is set.
    > > +It specifies the maximal number of 16-bit entries in RSS indirection
    > > table.
    > >
    > > -Field \field{rss_max_indirection_table_length} specifies the maximal
    > > number of 16-bit entries in RSS indirection table.
    > > +The next field, \field{supported_hash_types} exists only if the device
    > > supports hash calculation,
    > > +i.e. if VIRTIO_NET_F_RSS of VIRTIO_NET_F_HASH_REPORT is set.

    > of -> or?
    Yes, will fix the typo

    > >
    > > -Field \field{supported_hash_types} contains the bitmask of supported RSS
    > > hash types.
    > > -
    > > -Hash types applicable for IPv4 packets:
    > > -\begin{lstlisting}
    > > -#define VIRTIO_NET_RSS_HASH_TYPE_IPv4 (1 << 0)
    > > -#define VIRTIO_NET_RSS_HASH_TYPE_TCPv4 (1 << 1)
    > > -#define VIRTIO_NET_RSS_HASH_TYPE_UDPv4 (1 << 2)
    > > -\end{lstlisting}
    > > -Hash types applicable for IPv6 packets without extension headers
    > > -\begin{lstlisting}
    > > -#define VIRTIO_NET_RSS_HASH_TYPE_IPv6 (1 << 3)
    > > -#define VIRTIO_NET_RSS_HASH_TYPE_TCPv6 (1 << 4)
    > > -#define VIRTIO_NET_RSS_HASH_TYPE_UDPv6 (1 << 5)
    > > -\end{lstlisting}
    > > -Hash types applicable for IPv6 packets with extension headers
    > > -\begin{lstlisting}
    > > -#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX (1 << 6)
    > > -#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX (1 << 7)
    > > -#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX (1 << 8)
    > > -\end{lstlisting}
    > > -For the exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see
    > > \ref{sec:Device Types / Network Device / Device Operation / Control
    > > Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
    > > +Field \field{supported_hash_types} contains the bitmask of supported hash
    > > types.
    > > +See \ref{sec:Device Types / Network Device / Device Operation / Processing
    > > of Incoming Packets / Hash calculation for incoming packets /
    > > Supported/enabled hash types} for details of supported hash types.
    > >
    > > \devicenormative{\subsubsection}{Device configuration layout}{Device Types
    > > / Network Device / Device configuration layout}
    > >
    > > @@ -3034,7 +3019,7 @@ \subsection{Device configuration
    > > layout}\label{sec:Device Types / Network Device
    > > negotiated.
    > >
    > > The device MUST set \field{rss_max_key_size} to at least 40, if it offers
    > > -VIRTIO_NET_F_RSS.
    > > +VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
    > >
    > > The device MUST set \field{rss_max_indirection_table_length} to at least
    > > 128, if it offers
    > > VIRTIO_NET_F_RSS.
    > > @@ -3159,6 +3144,8 @@ \subsection{Device Operation}\label{sec:Device Types
    > > / Network Device / Device O
    > > le16 csum_start;
    > > le16 csum_offset;
    > > le16 num_buffers;
    > > + le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
    > > + le16 hash_type; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
    > > };
    > > \end{lstlisting}
    > >
    > > @@ -3481,6 +3468,12 @@ \subsubsection{Processing of Incoming
    > > Packets}\label{sec:Device Types / Network
    > >
    > > \end{enumerate}
    > >
    > > +If applicable, the device calculates per-packet hash for incoming packets
    > > as
    > > +defined in \ref{sec:Device Types / Network Device / Device Operation /
    > > Processing of Incoming Packets / Hash calculation for incoming packets}.
    > > +
    > > +If applicable, the device reports hash information for incoming packets as
    > > +defined in \ref{sec:Device Types / Network Device / Device Operation /
    > > Processing of Incoming Packets / Hash reporting for incoming packets}.
    > > +
    > > \devicenormative{\paragraph}{Processing of Incoming Packets}{Device Types /
    > > Network Device / Device Operation / Processing of Incoming Packets}
    > > \label{devicenormative:Device Types / Network Device / Device Operation /
    > > Processing of Packets}%old label for latexdiff
    > >
    > > @@ -3583,6 +3576,181 @@ \subsubsection{Processing of Incoming
    > > Packets}\label{sec:Device Types / Network
    > > If neither VIRTIO_NET_HDR_F_NEEDS_CSUM nor
    > > VIRTIO_NET_HDR_F_DATA_VALID is set, the driver MUST NOT
    > > rely on the packet checksum being correct.
    > > +
    > > +\paragraph{Hash calculation for incoming packets}
    > > +\label{sec:Device Types / Network Device / Device Operation / Processing
    > > of Incoming Packets / Hash calculation for incoming packets}
    > > +
    > > +A device attempts to calculate a per-packet hash in the following cases:
    > > +\begin{itemize}
    > > +\item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the
    > > hash to determine the receive virtqueue to place incoming packets.
    > > +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device
    > > reports the hash value and the hash type with the packet.
    > > +\end{itemize}

    > below mentions structures that were never mentioned earlier.
    > We should say a couple of words here.
    > E.g.
    > the driver configures the hash calculation by sending
    > either the virtio_net_rss_config structure as described in
    > ... or the
    > virtio_net_hash_config structure as described in ... to the device.

    OK, will add that

    > > +If the feature VIRTIO_NET_F_RSS was negotiated:
    > > +\begin{itemize}
    > > +\item The device uses \field{hash_types} of the virtio_net_rss_config
    > > structure as the 'Enabled hash types' bitmask.
    > > +\item The device uses \field{hash_key_data} of the virtio_net_rss_config
    > > structure.

    > .. as ... ?

    OK

    > Also what about key length field?
    > We should document all fields.

    > Also I would maybe repeat this near where the structure itself is defined.
    OK

    > > +\end{itemize}
    > > +See the definition of virtio_net_rss_config structure in \ref{sec:Device
    > > Types / Network Device / Device Operation / Control Virtqueue /
    > > Receive-side scaling (RSS) / Setting RSS parameters}

    > I would just add a link where virtio_net_rss_config is mentioned.
    > E.g.
    > The device uses \field{hash_key_data} of the virtio_net_rss_config
    > structure (See \ref{...}).

    OK

    > > +
    > > +If the feature VIRTIO_NET_F_RSS was not negotiated:
    > > +\begin{itemize}
    > > +\item The device uses \field{hash_types} of the virtio_net_hash_config
    > > structure as the 'Enabled hash types' bitmask.
    > > +\item The device uses \field{hash_key_data} of the virtio_net_hash_config
    > > structure.
    > > +\end{itemize}
    > > +See the definition of virtio_net_hash_config structure in \ref{sec:Device
    > > Types / Network Device / Device Operation / Control Virtqueue / Automatic
    > > receive steering in multiqueue mode / Hash calculation}.
    > > +
    > > +Note that if the device offers VIRTIO_NET_F_HASH_REPORT, even if it
    > > supports only one pair of virtqueues, it SHOULD support
    > > +at least one of commands of VIRTIO_NET_CTRL_MQ class to configure reported
    > > hash parameters:
    > > +\begin{itemize}
    > > +\item If the device offers VIRTIO_NET_F_RSS, it MUST support
    > > VIRTIO_NET_CTRL_MQ_RSS_CONFIG command per
    > > + \ref{sec:Device Types / Network Device / Device Operation / Control
    > > Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}.
    > > +\item Otherwise the device SHOULD support VIRTIO_NET_CTRL_MQ_HASH_CONFIG
    > > command per
    > > + \ref{sec:Device Types / Network Device / Device Operation / Control
    > > Virtqueue / Automatic receive steering in multiqueue mode / Hash
    > > calculation}.
    > > +\end{itemize}
    > > +
    > > +\subparagraph{Supported/enabled hash types}
    > > +\label{sec:Device Types / Network Device / Device Operation / Processing
    > > of Incoming Packets / Hash calculation for incoming packets /
    > > Supported/enabled hash types}
    > > +Hash types applicable for IPv4 packets:
    > > +\begin{lstlisting}
    > > +#define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
    > > +#define VIRTIO_NET_HASH_TYPE_TCPv4 (1 << 1)
    > > +#define VIRTIO_NET_HASH_TYPE_UDPv4 (1 << 2)
    > > +\end{lstlisting}
    > > +Hash types applicable for IPv6 packets without extension headers
    > > +\begin{lstlisting}
    > > +#define VIRTIO_NET_HASH_TYPE_IPv6 (1 << 3)
    > > +#define VIRTIO_NET_HASH_TYPE_TCPv6 (1 << 4)
    > > +#define VIRTIO_NET_HASH_TYPE_UDPv6 (1 << 5)
    > > +\end{lstlisting}
    > > +Hash types applicable for IPv6 packets with extension headers
    > > +\begin{lstlisting}
    > > +#define VIRTIO_NET_HASH_TYPE_IP_EX (1 << 6)
    > > +#define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
    > > +#define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
    > > +\end{lstlisting}
    > > +
    > > +\subparagraph{IPv4 packets}
    > > +\label{sec:Device Types / Network Device / Device Operation / Processing
    > > of Incoming Packets / Hash calculation for incoming packets / IPv4
    > > packets}
    > > +The device calculates the hash on IPv4 packets according to 'Enabled hash
    > > types' bitmask as follows:
    > > +\begin{itemize}
    > > +\item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the packet has
    > > +a TCP header, the hash is calculated over the following fields:
    > > +\begin{itemize}
    > > +\item Source IP address
    > > +\item Destination IP address
    > > +\item Source TCP port
    > > +\item Destination TCP port
    > > +\end{itemize}
    > > +\item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the
    > > +packet has a UDP header, the hash is calculated over the following fields:
    > > +\begin{itemize}
    > > +\item Source IP address
    > > +\item Destination IP address
    > > +\item Source UDP port
    > > +\item Destination UDP port
    > > +\end{itemize}
    > > +\item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is
    > > +calculated over the following fields:
    > > +\begin{itemize}
    > > +\item Source IP address
    > > +\item Destination IP address
    > > +\end{itemize}
    > > +\item Else the device does not calculate the hash
    > > +\end{itemize}
    > > +
    > > +\subparagraph{IPv6 packets without extension header}
    > > +\label{sec:Device Types / Network Device / Device Operation / Processing
    > > of Incoming Packets / Hash calculation for incoming packets / IPv6 packets
    > > without extension header}
    > > +The device calculates the hash on IPv6 packets without extension
    > > +headers according to 'Enabled hash types' bitmask as follows:
    > > +\begin{itemize}
    > > +\item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the packet has
    > > +a TCPv6 header, the hash is calculated over the following fields:
    > > +\begin{itemize}
    > > +\item Source IPv6 address
    > > +\item Destination IPv6 address
    > > +\item Source TCP port
    > > +\item Destination TCP port
    > > +\end{itemize}
    > > +\item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the
    > > +packet has a UDPv6 header, the hash is calculated over the following
    > > fields:
    > > +\begin{itemize}
    > > +\item Source IPv6 address
    > > +\item Destination IPv6 address
    > > +\item Source UDP port
    > > +\item Destination UDP port
    > > +\end{itemize}
    > > +\item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is
    > > +calculated over the following fields:
    > > +\begin{itemize}
    > > +\item Source IPv6 address
    > > +\item Destination IPv6 address
    > > +\end{itemize}
    > > +\item Else the device does not calculate the hash
    > > +\end{itemize}
    > > +
    > > +\subparagraph{IPv6 packets with extension header}
    > > +\label{sec:Device Types / Network Device / Device Operation / Processing
    > > of Incoming Packets / Hash calculation for incoming packets / IPv6 packets
    > > with extension header}
    > > +The device calculates the hash on IPv6 packets with extension
    > > +headers according to 'Enabled hash types' bitmask as follows:
    > > +\begin{itemize}
    > > +\item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the packet
    > > +has a TCPv6 header, the hash is calculated over the following fields:
    > > +\begin{itemize}
    > > +\item Home address from the home address option in the IPv6 destination
    > > options header. If the extension header is not present, use the Source
    > > IPv6 address.
    > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the
    > > associated extension header. If the extension header is not present, use
    > > the Destination IPv6 address.
    > > +\item Source TCP port
    > > +\item Destination TCP port
    > > +\end{itemize}
    > > +\item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the
    > > +packet has a UDPv6 header, the hash is calculated over the following
    > > fields:
    > > +\begin{itemize}
    > > +\item Home address from the home address option in the IPv6 destination
    > > options header. If the extension header is not present, use the Source
    > > IPv6 address.
    > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the
    > > associated extension header. If the extension header is not present, use
    > > the Destination IPv6 address.
    > > +\item Source UDP port
    > > +\item Destination UDP port
    > > +\end{itemize}
    > > +\item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is
    > > +calculated over the following fields:
    > > +\begin{itemize}
    > > +\item Home address from the home address option in the IPv6 destination
    > > options header. If the extension header is not present, use the Source
    > > IPv6 address.
    > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the
    > > associated extension header. If the extension header is not present, use
    > > the Destination IPv6 address.
    > > +\end{itemize}
    > > +\item Else skip IPv6 extension headers and calculate the hash as
    > > +defined for an IPv6 packet without extension headers
    > > +(see \ref{sec:Device Types / Network Device / Device Operation /
    > > Processing of Incoming Packets / Hash calculation for incoming packets /
    > > IPv6 packets without extension header}).
    > > +\end{itemize}
    > > +
    > > +\paragraph{Hash reporting for incoming packets}
    > > +\label{sec:Device Types / Network Device / Device Operation / Processing
    > > of Incoming Packets / Hash reporting for incoming packets}%old label for
    > > latexdiff

    > I doubt it's an old label

    OK, I'll remove that

    > > +
    > > +If feature VIRTIO_NET_F_HASH_REPORT

    > If feature VIRTIO_NET_F_HASH_REPORT -> If VIRTIO_NET_F_HASH_REPORT
    OK

    > > was negotiated and
    > > + the device has calculated the hash for the packet, the device fills
    > > \field{hash_type} with the type of calculated hash
    > > +and \field{hash_value} with the value of calculated hash.
    > > +
    > > +If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
    > > +hash was not calculated, the device sets \field{hash_type} to
    > > VIRTIO_NET_HASH_REPORT_NONE.
    > > +
    > > +Possible values that the device can report in \field{hash_type} are
    > > defined below.
    > > +They correspond to supported hash types defined in
    > > +\ref{sec:Device Types / Network Device / Device Operation / Processing of
    > > Incoming Packets / Hash calculation for incoming packets /
    > > Supported/enabled hash types}
    > > +as follows:
    > > +
    > > +VIRTIO_NET_HASH_TYPE_XXX = 1 << (VIRTIO_NET_HASH_REPORT_XXX - 1)
    > > +
    > > +\begin{lstlisting}
    > > +#define VIRTIO_NET_HASH_REPORT_NONE 0
    > > +#define VIRTIO_NET_HASH_REPORT_IPv4 1
    > > +#define VIRTIO_NET_HASH_REPORT_TCPv4 2
    > > +#define VIRTIO_NET_HASH_REPORT_UDPv4 3
    > > +#define VIRTIO_NET_HASH_REPORT_IPv6 4
    > > +#define VIRTIO_NET_HASH_REPORT_TCPv6 5
    > > +#define VIRTIO_NET_HASH_REPORT_UDPv6 6
    > > +#define VIRTIO_NET_HASH_REPORT_IPv6_EX 7
    > > +#define VIRTIO_NET_HASH_REPORT_TCPv6_EX 8
    > > +#define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
    > > +\end{lstlisting}
    > > +
    > > \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device /
    > > Device Operation / Control Virtqueue}
    > >
    > > The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
    > > @@ -3860,6 +4028,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device
    > > Types / Network Device / Devi
    > > #define VIRTIO_NET_CTRL_MQ 4
    > > #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET 0 (for automatic receive steering)
    > > #define VIRTIO_NET_CTRL_MQ_RSS_CONFIG 1 (for configurable receive steering)
    > > + #define VIRTIO_NET_CTRL_MQ_HASH_CONFIG 2 (for configurable hash
    > > calculation)
    > > \end{lstlisting}
    > >
    > > If more than one multiqueue mode is negotiated, the resulting device
    > > configuration is defined by the last command sent by the driver.
    > > @@ -3925,10 +4094,37 @@ \subsubsection{Control Virtqueue}\label{sec:Device
    > > Types / Network Device / Devi
    > > according to the native endian of the guest rather than
    > > (necessarily when not using the legacy interface) little-endian.
    > >
    > > +\subparagraph{Hash calculation}\label{sec:Device Types / Network Device /
    > > Device Operation / Control Virtqueue / Automatic receive steering in
    > > multiqueue mode / Hash calculation}
    > > +If VIRTIO_NET_F_HASH_REPORT negotiated

    > negotiated -> was negotiated

    OK

    > > and the device uses automatic receive steering,
    > > +the device SHOULD support a command to configure hash calculation
    > > parameters.

    > SHOULD and not MUST do drivers must expect failure and handle
    > gracefully?

    This corner non-functional case IMO does not present any critical problem to the driver.
    It already has negotiated the feature, so the packet is extended but does not contain a calculated hash.
    OK, I'll change it to MUST. Please respond if you disagree.

    > > +
    > > +The driver provides parameters for hash calculation as follows:
    > > +
    > > +\field{class} VIRTIO_NET_CTRL_MQ, \field{command}
    > > VIRTIO_NET_CTRL_MQ_HASH_CONFIG.
    > > +
    > > +The \field{command-specific-data} has following format:
    > > +\begin{lstlisting}
    > > +struct virtio_net_hash_config {
    > > + le32 hash_types;
    > > + le16 reserved[4];
    > > + u8 hash_key_length;
    > > + u8 hash_key_data[hash_key_length];
    > > +};
    > > +\end{lstlisting}
    > > +Field \field{hash_types} contains a bitmask of allowed hash types as
    > > +defined in
    > > +\ref{sec:Device Types / Network Device / Device Operation / Processing of
    > > Incoming Packets / Hash calculation for incoming packets /
    > > Supported/enabled hash types}.
    > > +Initially the device has all hash types disabled and reports only
    > > VIRTIO_NET_HASH_REPORT_NONE.
    > > +
    > > +Field \field{reserved} MUST contain zeroes. It is defined to make the
    > > structure to look similar

    > look similar -> match the layout of
    OK

    > > to virtio_net_rss_config structure,
    > > +defined in \ref{sec:Device Types / Network Device / Device Operation /
    > > Control Virtqueue / Receive-side scaling (RSS)}.
    > > +
    > > +Fields \field{hash_key_length} and \field{hash_key_data} define the key to
    > > be used in hash calculation.
    > > +
    > > \paragraph{Receive-side scaling (RSS)}\label{sec:Device Types / Network
    > > Device / Device Operation / Control Virtqueue / Receive-side scaling
    > > (RSS)}
    > > A device offers the feature VIRTIO_NET_F_RSS if it supports RSS receive
    > > steering with Toeplitz hash calculation and configurable parameters.
    > >
    > > -A driver queries RSS capabilities of the device by reading device
    > > configuration as defined in \ref{sec:Device Types / Network Device /
    > > Device configuration layout / RSS}
    > > +A driver queries RSS capabilities of the device by reading device
    > > configuration as defined in \ref{sec:Device Types / Network Device /
    > > Device configuration layout}
    > >
    > > \subparagraph{Setting RSS parameters}\label{sec:Device Types / Network
    > > Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)
    > > / Setting RSS parameters}
    > >
    > > @@ -3945,7 +4141,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device
    > > Types / Network Device / Devi
    > > };
    > > \end{lstlisting}
    > > Field \field{hash_types} contains a bitmask of allowed hash types as
    > > -defined in \ref{sec:Device Types / Network Device / Device Operation /
    > > Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
    > > +defined in
    > > +\ref{sec:Device Types / Network Device / Device Operation / Processing of
    > > Incoming Packets / Hash calculation for incoming packets /
    > > Supported/enabled hash types}.
    > >
    > > Field \field{indirection_table_mask} is a mask to be applied to
    > > the calculated hash to produce an index in the
    > > @@ -3959,94 +4156,6 @@ \subsubsection{Control Virtqueue}\label{sec:Device
    > > Types / Network Device / Devi
    > >
    > > A driver sets \field{max_tx_vq} to inform a device how many transmit
    > > virtqueues it may use (transmitq1\ldots transmitq \field{max_tx_vq}).
    > >
    > > -\subparagraph{RSS hash types}\label{sec:Device Types / Network Device /
    > > Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS
    > > hash types}
    > > -
    > > -The device calculates the hash on IPv4 packets according to the
    > > -field \field{hash_types} of the virtio_net_rss_config structure as
    > > follows:
    > > -\begin{itemize}
    > > -\item If VIRTIO_NET_RSS_HASH_TYPE_TCPv4 is set and the packet has
    > > -a TCP header, the hash is calculated over the following fields:
    > > -\begin{itemize}
    > > -\item Source IP address
    > > -\item Destination IP address
    > > -\item Source TCP port
    > > -\item Destination TCP port
    > > -\end{itemize}
    > > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDPv4 is set and the
    > > -packet has a UDP header, the hash is calculated over the following fields:
    > > -\begin{itemize}
    > > -\item Source IP address
    > > -\item Destination IP address
    > > -\item Source UDP port
    > > -\item Destination UDP port
    > > -\end{itemize}
    > > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IPv4 is set, the hash is
    > > -calculated over the following fields:
    > > -\begin{itemize}
    > > -\item Source IP address
    > > -\item Destination IP address
    > > -\end{itemize}
    > > -\item Else the device does not calculate the hash
    > > -\end{itemize}
    > > -
    > > -\label{itm:Device Types / Network Device / Device Operation / Control
    > > Virtqueue / Receive-side scaling (RSS) / RSS hash types/hash on IPv6
    > > packets without extension headers}
    > > -The device calculates the hash on IPv6 packets without extension
    > > -headers according to the field \field{hash_types} of the
    > > virtio_net_rss_config structure as follows:
    > > -\begin{itemize}
    > > -\item If VIRTIO_NET_RSS_HASH_TYPE_TCPv6 is set and the packet has
    > > -a TCPv6 header, the hash is calculated over the following fields:
    > > -\begin{itemize}
    > > -\item Source IPv6 address
    > > -\item Destination IPv6 address
    > > -\item Source TCP port
    > > -\item Destination TCP port
    > > -\end{itemize}
    > > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDPv6 is set and the
    > > -packet has a UDPv6 header, the hash is calculated over the following
    > > fields:
    > > -\begin{itemize}
    > > -\item Source IPv6 address
    > > -\item Destination IPv6 address
    > > -\item Source UDP port
    > > -\item Destination UDP port
    > > -\end{itemize}
    > > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IPv6 is set, the hash is
    > > -calculated over the following fields:
    > > -\begin{itemize}
    > > -\item Source IPv6 address
    > > -\item Destination IPv6 address
    > > -\end{itemize}
    > > -\item Else the device does not calculate the hash
    > > -\end{itemize}
    > > -
    > > -The device calculates the hash on IPv6 packets with extension
    > > -headers according to the field \field{hash_types} of the
    > > virtio_net_rss_config structure as follows:
    > > -\begin{itemize}
    > > -\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet
    > > -has a TCPv6 header, the hash is calculated over the following fields:
    > > -\begin{itemize}
    > > -\item Home address from the home address option in the IPv6 destination
    > > options header. If the extension header is not present, use the Source
    > > IPv6 address.
    > > -\item IPv6 address that is contained in the Routing-Header-Type-2 from the
    > > associated extension header. If the extension header is not present, use
    > > the Destination IPv6 address.
    > > -\item Source TCP port
    > > -\item Destination TCP port
    > > -\end{itemize}
    > > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the
    > > -packet has a UDPv6 header, the hash is calculated over the following
    > > fields:
    > > -\begin{itemize}
    > > -\item Home address from the home address option in the IPv6 destination
    > > options header. If the extension header is not present, use the Source
    > > IPv6 address.
    > > -\item IPv6 address that is contained in the Routing-Header-Type-2 from the
    > > associated extension header. If the extension header is not present, use
    > > the Destination IPv6 address.
    > > -\item Source UDP port
    > > -\item Destination UDP port
    > > -\end{itemize}
    > > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is
    > > -calculated over the following fields:
    > > -\begin{itemize}
    > > -\item Home address from the home address option in the IPv6 destination
    > > options header. If the extension header is not present, use the Source
    > > IPv6 address.
    > > -\item IPv6 address that is contained in the Routing-Header-Type-2 from the
    > > associated extension header. If the extension header is not present, use
    > > the Destination IPv6 address.
    > > -\end{itemize}
    > > -\item Else skip IPv6 extension headers and calculate the hash as
    > > -defined for an IPv6 packet without extension headers (see \ref{itm:Device
    > > Types / Network Device / Device Operation / Control Virtqueue /
    > > Receive-side scaling (RSS) / RSS hash types/hash on IPv6 packets without
    > > extension headers}).
    > > -\end{itemize}
    > > -
    > > \drivernormative{\subparagraph}{Setting RSS parameters}{Device Types /
    > > Network Device / Device Operation / Control Virtqueue / Receive-side
    > > scaling (RSS) }
    > >
    > > A driver MUST NOT send the VIRTIO_NET_CTRL_MQ_RSS_CONFIG command if the
    > > feature VIRTIO_NET_F_RSS has not been negotiated.
    > > @@ -4057,12 +4166,12 @@ \subsubsection{Control Virtqueue}\label{sec:Device
    > > Types / Network Device / Devi
    > >
    > > A driver MUST use \field{indirection_table_mask} values that are less than
    > > \field{rss_max_indirection_table_length} reported by a device.
    > >
    > > -A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not
    > > supported by a device.
    > > +A driver MUST NOT set any VIRTIO_NET_HASH_TYPE_ flags that are not
    > > supported by a device.
    > >
    > > \devicenormative{\subparagraph}{RSS processing}{Device Types / Network
    > > Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)
    > > / RSS processing}
    > > The device MUST determine the destination queue for a network packet as
    > > follows:
    > > \begin{itemize}
    > > -\item Calculate the hash of the packet as defined in \ref{sec:Device Types
    > > / Network Device / Device Operation / Control Virtqueue / Receive-side
    > > scaling (RSS) / RSS hash types}
    > > +\item Calculate the hash of the packet as defined in \ref{sec:Device Types
    > > / Network Device / Device Operation / Processing of Incoming Packets /
    > > Hash calculation for incoming packets}.
    > > \item If the device did not calculate the hash for the specific packet, the
    > > device directs the packet to the receiveq specified by
    > > \field{unclassified_queue} of virtio_net_rss_config structure (value of 0
    > > corresponds to receiveq1).
    > > \item Apply \field{indirection_table_mask} to the calculated hash and use
    > > the result as the index in the indirection table to get 0-based number of
    > > destination receiveq (value of 0 corresponds to receiveq1).
    > > \end{itemize}
    > > --

    I'll prepare the next drop with these minimal changes.

    > > 2.17.1



  • 4.  Re: [virtio-comment] [PATCH v3] virtio-net: Define per-packet hash reporting feature

    Posted 02-19-2020 07:45
    On Wed, Feb 05, 2020 at 08:59:05PM +0200, Yuri Benditovich wrote:
    > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/66
    > Define respective feature bit for virtio-net.
    > Extend packet layout to populate hash value and type.
    > Move the definition of IP/TCP/UDP header fields to
    > calculate the hash out of RSS section to common network
    > device section.
    >
    > Signed-off-by: Yuri Benditovich <yuri.benditovich@daynix.com>
    > ---
    > content.tex | 347 ++++++++++++++++++++++++++++++++++------------------
    > 1 file changed, 228 insertions(+), 119 deletions(-)
    >
    > diff --git a/content.tex b/content.tex
    > index fb10061..20d598d 100644
    > --- a/content.tex
    > +++ b/content.tex
    > @@ -2895,6 +2895,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
    > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
    > channel.
    >
    > +\item[VIRTIO_NET_F_HASH_REPORT(58)] Device can report per-packet hash
    > + value and a type of calculated hash.
    > +
    > \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
    > value. Device benefits from knowing the exact header length.
    >
    > @@ -2981,35 +2984,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
    > le32 supported_hash_types;
    > };
    > \end{lstlisting}
    > -\label{sec:Device Types / Network Device / Device configuration layout / RSS}
    > -Three following fields, \field{rss_max_key_size}, \field{rss_max_indirection_table_length}
    > -and \field{supported_hash_types} only exist if VIRTIO_NET_F_RSS is set.
    > +The following field, \field{rss_max_key_size} only exist if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
    > +It specifies the maximal supported length of RSS key in bytes.
    >
    > -Field \field{rss_max_key_size} specifies the maximal supported length of RSS key in bytes.
    > +The following field, \field{rss_max_indirection_table_length} only exist if VIRTIO_NET_F_RSS is set.
    > +It specifies the maximal number of 16-bit entries in RSS indirection table.
    >
    > -Field \field{rss_max_indirection_table_length} specifies the maximal number of 16-bit entries in RSS indirection table.
    > +The next field, \field{supported_hash_types} exists only if the device supports hash calculation,
    > +i.e. if VIRTIO_NET_F_RSS of VIRTIO_NET_F_HASH_REPORT is set.
    >
    > -Field \field{supported_hash_types} contains the bitmask of supported RSS hash types.
    > -
    > -Hash types applicable for IPv4 packets:
    > -\begin{lstlisting}
    > -#define VIRTIO_NET_RSS_HASH_TYPE_IPv4 (1 << 0)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_TCPv4 (1 << 1)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_UDPv4 (1 << 2)
    > -\end{lstlisting}
    > -Hash types applicable for IPv6 packets without extension headers
    > -\begin{lstlisting}
    > -#define VIRTIO_NET_RSS_HASH_TYPE_IPv6 (1 << 3)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_TCPv6 (1 << 4)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_UDPv6 (1 << 5)
    > -\end{lstlisting}
    > -Hash types applicable for IPv6 packets with extension headers
    > -\begin{lstlisting}
    > -#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX (1 << 6)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX (1 << 7)
    > -#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX (1 << 8)
    > -\end{lstlisting}
    > -For the exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
    > +Field \field{supported_hash_types} contains the bitmask of supported hash types.
    > +See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} for details of supported hash types.
    >
    > \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
    >
    > @@ -3034,7 +3019,7 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
    > negotiated.
    >
    > The device MUST set \field{rss_max_key_size} to at least 40, if it offers
    > -VIRTIO_NET_F_RSS.
    > +VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
    >
    > The device MUST set \field{rss_max_indirection_table_length} to at least 128, if it offers
    > VIRTIO_NET_F_RSS.
    > @@ -3159,6 +3144,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
    > le16 csum_start;
    > le16 csum_offset;
    > le16 num_buffers;
    > + le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
    > + le16 hash_type; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)


    BTW I suspect we need to reserve 2 more bytes to avoid alignment
    problems..

    > };
    > \end{lstlisting}
    >
    > @@ -3481,6 +3468,12 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
    >
    > \end{enumerate}
    >
    > +If applicable, the device calculates per-packet hash for incoming packets as
    > +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets}.
    > +
    > +If applicable, the device reports hash information for incoming packets as
    > +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}.
    > +
    > \devicenormative{\paragraph}{Processing of Incoming Packets}{Device Types / Network Device / Device Operation / Processing of Incoming Packets}
    > \label{devicenormative:Device Types / Network Device / Device Operation / Processing of Packets}%old label for latexdiff
    >
    > @@ -3583,6 +3576,181 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
    > If neither VIRTIO_NET_HDR_F_NEEDS_CSUM nor
    > VIRTIO_NET_HDR_F_DATA_VALID is set, the driver MUST NOT
    > rely on the packet checksum being correct.
    > +
    > +\paragraph{Hash calculation for incoming packets}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets}
    > +
    > +A device attempts to calculate a per-packet hash in the following cases:
    > +\begin{itemize}
    > +\item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
    > +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
    > +\end{itemize}
    > +
    > +If the feature VIRTIO_NET_F_RSS was negotiated:
    > +\begin{itemize}
    > +\item The device uses \field{hash_types} of the virtio_net_rss_config structure as the 'Enabled hash types' bitmask.
    > +\item The device uses \field{hash_key_data} of the virtio_net_rss_config structure.
    > +\end{itemize}
    > +See the definition of virtio_net_rss_config structure in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
    > +
    > +If the feature VIRTIO_NET_F_RSS was not negotiated:
    > +\begin{itemize}
    > +\item The device uses \field{hash_types} of the virtio_net_hash_config structure as the 'Enabled hash types' bitmask.
    > +\item The device uses \field{hash_key_data} of the virtio_net_hash_config structure.
    > +\end{itemize}
    > +See the definition of virtio_net_hash_config structure in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
    > +
    > +Note that if the device offers VIRTIO_NET_F_HASH_REPORT, even if it supports only one pair of virtqueues, it SHOULD support
    > +at least one of commands of VIRTIO_NET_CTRL_MQ class to configure reported hash parameters:
    > +\begin{itemize}
    > +\item If the device offers VIRTIO_NET_F_RSS, it MUST support VIRTIO_NET_CTRL_MQ_RSS_CONFIG command per
    > + \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}.
    > +\item Otherwise the device SHOULD support VIRTIO_NET_CTRL_MQ_HASH_CONFIG command per
    > + \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
    > +\end{itemize}
    > +
    > +\subparagraph{Supported/enabled hash types}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
    > +Hash types applicable for IPv4 packets:
    > +\begin{lstlisting}
    > +#define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
    > +#define VIRTIO_NET_HASH_TYPE_TCPv4 (1 << 1)
    > +#define VIRTIO_NET_HASH_TYPE_UDPv4 (1 << 2)
    > +\end{lstlisting}
    > +Hash types applicable for IPv6 packets without extension headers
    > +\begin{lstlisting}
    > +#define VIRTIO_NET_HASH_TYPE_IPv6 (1 << 3)
    > +#define VIRTIO_NET_HASH_TYPE_TCPv6 (1 << 4)
    > +#define VIRTIO_NET_HASH_TYPE_UDPv6 (1 << 5)
    > +\end{lstlisting}
    > +Hash types applicable for IPv6 packets with extension headers
    > +\begin{lstlisting}
    > +#define VIRTIO_NET_HASH_TYPE_IP_EX (1 << 6)
    > +#define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
    > +#define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
    > +\end{lstlisting}
    > +
    > +\subparagraph{IPv4 packets}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
    > +The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
    > +\begin{itemize}
    > +\item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the packet has
    > +a TCP header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IP address
    > +\item Destination IP address
    > +\item Source TCP port
    > +\item Destination TCP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the
    > +packet has a UDP header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IP address
    > +\item Destination IP address
    > +\item Source UDP port
    > +\item Destination UDP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is
    > +calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IP address
    > +\item Destination IP address
    > +\end{itemize}
    > +\item Else the device does not calculate the hash
    > +\end{itemize}
    > +
    > +\subparagraph{IPv6 packets without extension header}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}
    > +The device calculates the hash on IPv6 packets without extension
    > +headers according to 'Enabled hash types' bitmask as follows:
    > +\begin{itemize}
    > +\item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the packet has
    > +a TCPv6 header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IPv6 address
    > +\item Destination IPv6 address
    > +\item Source TCP port
    > +\item Destination TCP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the
    > +packet has a UDPv6 header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IPv6 address
    > +\item Destination IPv6 address
    > +\item Source UDP port
    > +\item Destination UDP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is
    > +calculated over the following fields:
    > +\begin{itemize}
    > +\item Source IPv6 address
    > +\item Destination IPv6 address
    > +\end{itemize}
    > +\item Else the device does not calculate the hash
    > +\end{itemize}
    > +
    > +\subparagraph{IPv6 packets with extension header}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets with extension header}
    > +The device calculates the hash on IPv6 packets with extension
    > +headers according to 'Enabled hash types' bitmask as follows:
    > +\begin{itemize}
    > +\item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the packet
    > +has a TCPv6 header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > +\item Source TCP port
    > +\item Destination TCP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the
    > +packet has a UDPv6 header, the hash is calculated over the following fields:
    > +\begin{itemize}
    > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > +\item Source UDP port
    > +\item Destination UDP port
    > +\end{itemize}
    > +\item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is
    > +calculated over the following fields:
    > +\begin{itemize}
    > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > +\end{itemize}
    > +\item Else skip IPv6 extension headers and calculate the hash as
    > +defined for an IPv6 packet without extension headers
    > +(see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
    > +\end{itemize}
    > +
    > +\paragraph{Hash reporting for incoming packets}
    > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}%old label for latexdiff
    > +
    > +If feature VIRTIO_NET_F_HASH_REPORT was negotiated and
    > + the device has calculated the hash for the packet, the device fills \field{hash_type} with the type of calculated hash
    > +and \field{hash_value} with the value of calculated hash.
    > +
    > +If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
    > +hash was not calculated, the device sets \field{hash_type} to VIRTIO_NET_HASH_REPORT_NONE.
    > +
    > +Possible values that the device can report in \field{hash_type} are defined below.
    > +They correspond to supported hash types defined in
    > +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
    > +as follows:
    > +
    > +VIRTIO_NET_HASH_TYPE_XXX = 1 << (VIRTIO_NET_HASH_REPORT_XXX - 1)
    > +
    > +\begin{lstlisting}
    > +#define VIRTIO_NET_HASH_REPORT_NONE 0
    > +#define VIRTIO_NET_HASH_REPORT_IPv4 1
    > +#define VIRTIO_NET_HASH_REPORT_TCPv4 2
    > +#define VIRTIO_NET_HASH_REPORT_UDPv4 3
    > +#define VIRTIO_NET_HASH_REPORT_IPv6 4
    > +#define VIRTIO_NET_HASH_REPORT_TCPv6 5
    > +#define VIRTIO_NET_HASH_REPORT_UDPv6 6
    > +#define VIRTIO_NET_HASH_REPORT_IPv6_EX 7
    > +#define VIRTIO_NET_HASH_REPORT_TCPv6_EX 8
    > +#define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
    > +\end{lstlisting}
    > +
    > \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
    >
    > The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
    > @@ -3860,6 +4028,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    > #define VIRTIO_NET_CTRL_MQ 4
    > #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET 0 (for automatic receive steering)
    > #define VIRTIO_NET_CTRL_MQ_RSS_CONFIG 1 (for configurable receive steering)
    > + #define VIRTIO_NET_CTRL_MQ_HASH_CONFIG 2 (for configurable hash calculation)
    > \end{lstlisting}
    >
    > If more than one multiqueue mode is negotiated, the resulting device configuration is defined by the last command sent by the driver.
    > @@ -3925,10 +4094,37 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    > according to the native endian of the guest rather than
    > (necessarily when not using the legacy interface) little-endian.
    >
    > +\subparagraph{Hash calculation}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}
    > +If VIRTIO_NET_F_HASH_REPORT negotiated and the device uses automatic receive steering,
    > +the device SHOULD support a command to configure hash calculation parameters.
    > +
    > +The driver provides parameters for hash calculation as follows:
    > +
    > +\field{class} VIRTIO_NET_CTRL_MQ, \field{command} VIRTIO_NET_CTRL_MQ_HASH_CONFIG.
    > +
    > +The \field{command-specific-data} has following format:
    > +\begin{lstlisting}
    > +struct virtio_net_hash_config {
    > + le32 hash_types;
    > + le16 reserved[4];
    > + u8 hash_key_length;
    > + u8 hash_key_data[hash_key_length];
    > +};
    > +\end{lstlisting}
    > +Field \field{hash_types} contains a bitmask of allowed hash types as
    > +defined in
    > +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
    > +Initially the device has all hash types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
    > +
    > +Field \field{reserved} MUST contain zeroes. It is defined to make the structure to look similar to virtio_net_rss_config structure,
    > +defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}.
    > +
    > +Fields \field{hash_key_length} and \field{hash_key_data} define the key to be used in hash calculation.
    > +
    > \paragraph{Receive-side scaling (RSS)}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}
    > A device offers the feature VIRTIO_NET_F_RSS if it supports RSS receive steering with Toeplitz hash calculation and configurable parameters.
    >
    > -A driver queries RSS capabilities of the device by reading device configuration as defined in \ref{sec:Device Types / Network Device / Device configuration layout / RSS}
    > +A driver queries RSS capabilities of the device by reading device configuration as defined in \ref{sec:Device Types / Network Device / Device configuration layout}
    >
    > \subparagraph{Setting RSS parameters}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
    >
    > @@ -3945,7 +4141,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    > };
    > \end{lstlisting}
    > Field \field{hash_types} contains a bitmask of allowed hash types as
    > -defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
    > +defined in
    > +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
    >
    > Field \field{indirection_table_mask} is a mask to be applied to
    > the calculated hash to produce an index in the
    > @@ -3959,94 +4156,6 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    >
    > A driver sets \field{max_tx_vq} to inform a device how many transmit virtqueues it may use (transmitq1\ldots transmitq \field{max_tx_vq}).
    >
    > -\subparagraph{RSS hash types}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
    > -
    > -The device calculates the hash on IPv4 packets according to the
    > -field \field{hash_types} of the virtio_net_rss_config structure as follows:
    > -\begin{itemize}
    > -\item If VIRTIO_NET_RSS_HASH_TYPE_TCPv4 is set and the packet has
    > -a TCP header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IP address
    > -\item Destination IP address
    > -\item Source TCP port
    > -\item Destination TCP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDPv4 is set and the
    > -packet has a UDP header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IP address
    > -\item Destination IP address
    > -\item Source UDP port
    > -\item Destination UDP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IPv4 is set, the hash is
    > -calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IP address
    > -\item Destination IP address
    > -\end{itemize}
    > -\item Else the device does not calculate the hash
    > -\end{itemize}
    > -
    > -\label{itm:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types/hash on IPv6 packets without extension headers}
    > -The device calculates the hash on IPv6 packets without extension
    > -headers according to the field \field{hash_types} of the virtio_net_rss_config structure as follows:
    > -\begin{itemize}
    > -\item If VIRTIO_NET_RSS_HASH_TYPE_TCPv6 is set and the packet has
    > -a TCPv6 header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IPv6 address
    > -\item Destination IPv6 address
    > -\item Source TCP port
    > -\item Destination TCP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDPv6 is set and the
    > -packet has a UDPv6 header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IPv6 address
    > -\item Destination IPv6 address
    > -\item Source UDP port
    > -\item Destination UDP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IPv6 is set, the hash is
    > -calculated over the following fields:
    > -\begin{itemize}
    > -\item Source IPv6 address
    > -\item Destination IPv6 address
    > -\end{itemize}
    > -\item Else the device does not calculate the hash
    > -\end{itemize}
    > -
    > -The device calculates the hash on IPv6 packets with extension
    > -headers according to the field \field{hash_types} of the virtio_net_rss_config structure as follows:
    > -\begin{itemize}
    > -\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet
    > -has a TCPv6 header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > -\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > -\item Source TCP port
    > -\item Destination TCP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the
    > -packet has a UDPv6 header, the hash is calculated over the following fields:
    > -\begin{itemize}
    > -\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > -\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > -\item Source UDP port
    > -\item Destination UDP port
    > -\end{itemize}
    > -\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is
    > -calculated over the following fields:
    > -\begin{itemize}
    > -\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
    > -\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
    > -\end{itemize}
    > -\item Else skip IPv6 extension headers and calculate the hash as
    > -defined for an IPv6 packet without extension headers (see \ref{itm:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types/hash on IPv6 packets without extension headers}).
    > -\end{itemize}
    > -
    > \drivernormative{\subparagraph}{Setting RSS parameters}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) }
    >
    > A driver MUST NOT send the VIRTIO_NET_CTRL_MQ_RSS_CONFIG command if the feature VIRTIO_NET_F_RSS has not been negotiated.
    > @@ -4057,12 +4166,12 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
    >
    > A driver MUST use \field{indirection_table_mask} values that are less than \field{rss_max_indirection_table_length} reported by a device.
    >
    > -A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not supported by a device.
    > +A driver MUST NOT set any VIRTIO_NET_HASH_TYPE_ flags that are not supported by a device.
    >
    > \devicenormative{\subparagraph}{RSS processing}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
    > The device MUST determine the destination queue for a network packet as follows:
    > \begin{itemize}
    > -\item Calculate the hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
    > +\item Calculate the hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets}.
    > \item If the device did not calculate the hash for the specific packet, the device directs the packet to the receiveq specified by \field{unclassified_queue} of virtio_net_rss_config structure (value of 0 corresponds to receiveq1).
    > \item Apply \field{indirection_table_mask} to the calculated hash and use the result as the index in the indirection table to get 0-based number of destination receiveq (value of 0 corresponds to receiveq1).
    > \end{itemize}
    > --
    > 2.17.1
    >
    >
    > This publicly archived list offers a means to provide input to the
    > OASIS Virtual I/O Device (VIRTIO) TC.
    >
    > In order to verify user consent to the Feedback License terms and
    > to minimize spam in the list archive, subscription is required
    > before posting.
    >
    > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
    > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
    > List help: virtio-comment-help@lists.oasis-open.org
    > List archive: https://lists.oasis-open.org/archives/virtio-comment/
    > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
    > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
    > Committee: https://www.oasis-open.org/committees/virtio/
    > Join OASIS: https://www.oasis-open.org/join/