Thanks, Bret for clarifying on Slack; I retract the below. I was obviously unclear on taxii 2.1 dropping the range headers completely.
From: Bret Jordan <
Bret_Jordan@symantec.com>
Date: Friday, October 25, 2019 at 15:08
To: Matt Pladna <
mpladna@lookingglasscyber.com>, "cti@lists.oasis-open.org" <
cti@lists.oasis-open.org>
Subject: Re: [EXT] Re: [cti] TAXII 2.1 WD 09 reminder
Matt,
The Range headers were completely dropped from 2.1, that was done way back in Working Draft 01.
Bret
From: Matt Pladna <
mpladna@lookingglasscyber.com>
Sent: Friday, October 25, 2019 12:42 PM
To: Bret Jordan <
Bret_Jordan@symantec.com>;
cti@lists.oasis-open.org <
cti@lists.oasis-open.org>
Subject: [EXT] Re: [cti] TAXII 2.1 WD 09 reminder
Bret,
Thanks for the reminder. I reviewed the pagination text in WD 09. I don t believe it s clear enough about the two pagination methods being allowed: Range/Content-Range headers from 2.0 and Limit Parameter in 2.1. I took a stab at drafting
up some text below. Note that my headers are probably not sized right for be sub sections.
I d appreciate you and a couple others reviewing and weighing in. I ve modified slightly the main pagination section and added subsections to clearly split the range pagination from limit param.
3.5 Pagination
TAXII 2.1 supports pagination of large result sets on certain endpoints. These endpoints return results sorted in ascending order by the date they were added to the collection (see
section 3.3 ).
The server may limit the number of responses in response to a query in one of two ways:
The server can use HTTP's
Range and
Content-Range (Range/Content-Range Pagination) headers, and defines the
items range unit, to perform pagination as defined in section 2 of [ RFC7233 ].
items is the mandatory to implement range unit for TAXII. Other range units MAY be implemented
by clients and servers.
A limit parameter (Limit parameter
Pagination) can be used by a client for a request to a paginated endpoint (see section
3.4 ).
If more objects are available after a paginated response, either because the client requested that they be limited via the
limit parameter or the client/server limits them via
Range and
Content-Range headers, then the response
envelope
MUST have a value of true in the
more property and
MAY have an appropriate value in the next property.
If the
more property is set to
true and the
next property is populated then the client can paginate through the remaining records using the
next URL parameter along with the same original query options.
If the
more property is set to
true and the
next property is empty then the client may paginate through the remaining records by
using the added_after URL parameter
with the date/time value from the X-TAXII-Date-Added-Last header along with the same original query
options.
Using Range and
Content-Range headers
It is possible for the server to return
"more": true in a response, yet present no additional objects in the follow-on query.
This can occur when the additional objects are deleted from a collection between requests.
3.5.1 Range/Content-Range Pagination
The
items range unit is defined for expressing subranges of a resource [ HTTP
7233 ]. For the Endpoints that return
object ,
items represents objects. For the Endpoints that return
collections ,
items represents Collections.
The first
items value in the
Range and
Content-Range headers gives the first item in a range. The last
items value in the
Range and
Content-Range headers gives the last item in the range; that is,
items ranges specified are inclusive.
items are zero-indexed; that is, the first item is object zero. A
Content-Range header will have a third value that identifies the size of the available
dataset.
For example:
If the Range header contains " items
10-49", "10" represents the first item requested; and "49" represents the last item requested.
if the Content-Range header contains
" items 10-49/500", "10" represents the first object in the response; "49" represents the last object
in the response; and "500" represents the total number of items available.
All
items values
MUST be:
a non-negative
integer
zero indexed (i.e., the first object is object "0")
3.5.1.1 Range/Content-Range Requirements
The following requirements only apply to
items based range requests
The Accept-Ranges header allows a
server to indicate that it supports range requests for the target resource [ RFC7233 ]
as well as which range units are supported.
For resources where
items -based pagination is supported, and where the
Accept-Ranges header is present, the
Accept-Ranges header
MUST contain items as an acceptable range. The
Accept-Ranges header
MAY contain other acceptable ranges, if the server supports them.
Requests MAY use the
Range header to request a subset of data that would otherwise be returned.
As defined in the HTTP specification, HTTP 206 (Partial Content) [ RFC7233 ]
responses include a Content-Range header, even if the entire resource is contained in the response.
As defined in the HTTP specification, if the requested
Range cannot be satisfied, an HTTP 416 (Requested Range Not Satisfiable) [ RFC7233 ]
response is used.
For example, if a range requests
items 500-600, but only 0-100 are available, an HTTP 416 (Requested Range Not Satisfiable) is used.
An HTTP 206 (Partial Content) response with a
Content-Range header
MAY be returned even if the original request did not have a Range header.
Note that this is a deviation from the HTTP specification, which indicates that HTTP 206 responses are only permitted when the
Range header is present in the request.
Responses to requests with a
Range header
SHOULD contain only the requested range and MAY include a range smaller than what was requested.
TAXII follows standard HTTP rules for the
Content-Range and
Range headers, with the exception of allowing a 206 response to a request without a
Range header:
The 206 (Partial Content) status code indicates that the server is successfully fulfilling a range request for the target resource see section 4.1 of [ RFC7233 ]
If a single part is being transferred, the server generating the 206 response
MUST generate a Content-Range header field, describing what range of the selected representation
is enclosed, and a payload consisting of the range. See section 4.2 of [ RFC7233 ].
NOTE: The total number of items available in a result may change with each request for a new page in the paginated result set. This can happen if items have been added or deleted between
subsequent requests.
3.5.2 Limit Parameter Pagination
Collection High-Value-Indicators has 1000 records in it.
The client or server has limited all responses to 100 records at a time.
A client will make a request and the server will respond with the first 100 records.
The server will also populate the two X headers for TAXII,
X-TAXII-Date-Added-First and
X-TAXII-Date-Added-Last . These headers will contain the date/time value of when the first and last
records of the 100 returned records were added to the TAXII server.
The server will also set the
more property to a value of
true and may set the
next property to an appropriate value on the TAXII
envelope .
When a client wants to obtain the next 100 records, the client will do either:
Provided the same query/filter parameters that it originally used along with populating the
next URL parameter with the value from the
next property on the previous TAXII
envelope response.
Or provide the same query/filter parameters that it originally used along with the date/time value from the
X-TAXII-Date-Added-Last in the
added_after URL parameter
From:
<
cti@lists.oasis-open.org> on behalf of Bret Jordan <
Bret_Jordan@symantec.com>
Date: Friday, October 25, 2019 at 13:31
To: "cti@lists.oasis-open.org" <
cti@lists.oasis-open.org>
Subject: [cti] TAXII 2.1 WD 09 reminder
All,
Just a reminder that all comments and suggestions for TAXII 2.1 WD09 are due today.
Thanks
Bret