I agree w/ Allan s criteria. Specifically:
IMO new endpoints should almost always require the process. It s a check against adding a ton of new endpoints on a whim without doing testing to make sure they re implementable and
that you can have sane interoperability text. Maybe this is pointing out the obvious, but the criteria that non-compatible changes go through the process could mean applying it to 2.1, since some major features were changed and
removed. Some of those changes are very straightforward and probably wouldn t be helped with testing (e.g. MIME type change). Others maybe it does make sense to do this for for example, pagination issues were only caught once people tried to implement it.
It s probably worth going through the process for this set of changes before finalizing the CS in order to avoid yet another breaking change to pagination (my impression is that at least one implementation is done already so this doesn t necessarily need to
add a ton of delays).
John
From: <
cti@lists.oasis-open.org> on behalf of "Bret Jordan (CS)" <
Bret_Jordan@symantec.com>
Date: Monday, December 10, 2018 at 10:54 AM
To: Allan Thomson <
athomson@lookingglasscyber.com>, John-Mark Gurney <
jmg@newcontext.com>, Matt Pladna <
mpladna@lookingglasscyber.com>
Cc: "Kelley, Sarah E." <
skelley@mitre.org>, "cti@lists.oasis-open.org" <
cti@lists.oasis-open.org>
Subject: Re: [cti] RE: [EXT] Re: [cti] TAXII definition of "Done"
Thank you for the comments and feedback. I like it.
Bret
From: Allan Thomson <
athomson@lookingglasscyber.com>
Sent: Monday, December 10, 2018 8:16:00 AM
To: Bret Jordan; John-Mark Gurney; Matt Pladna
Cc: Kelley, Sarah E.;
cti@lists.oasis-open.org Subject: Re: [cti] RE: [EXT] Re: [cti] TAXII definition of "Done"
Bret here s some input on the TAXII criteria
Two sponsors
AT: At least 1 TXS and 1 TAXII Client (different orgs). If we wanted to be consistent with STIX then it should really be 2 TXS and at least 1 TAXII Client.
Needs to be implemented in two different code bases
Agreed with at least 1 on server-side and 1 on client-side from different orgs
Needs to have proof-of-concept code
Agreed on both server and client sides.
Needs all specification text and some interoperability tests
Agreed (on both server and client sides).
Q: What constitutes a new feature?
AT: This is somewhat subjective based on the change but here some suggestions:
1.
One or more new additional endpoints/methods to support change
2.
Non-compatible (forward or backward) change to previous version of TAXII
3.
Additional semantic behavior on existing method/endpoints (with possible additional arguments to support) that is more complex than a few lines of code to support
Allan Thomson
CTO ( +1-408-331-6646)
LookingGlass
Cyber Solutions
From:
"cti@lists.oasis-open.org" <
cti@lists.oasis-open.org> on behalf of Bret Jordan <
Bret_Jordan@symantec.com>
Date: Saturday, December 8, 2018 at 1:51 PM
To: John-Mark Gurney <
jmg@newcontext.com>, Matt Pladna <
mpladna@lookingglasscyber.com>
Cc: "Kelley, Sarah E." <
skelley@mitre.org>, "cti@lists.oasis-open.org" <
cti@lists.oasis-open.org>
Subject: Re: [cti] RE: [EXT] Re: [cti] TAXII definition of "Done"
So for STIX we have a rule system that new features need:
1) Two sponsors
2) Needs to be implemented in two different code bases
3) Needs to have proof-of-concept code
4) Needs all specification text and some interoperability tests
A question for TAXII is, is this rule set still applicable? Or does it need some changes? If so, what constitutes a new feature? In STIX this is generally understood to mean a new object. but what is the equivalent
in TAXII. Also additional properties, changes to properties, new relationship types, new vocab entries do not have a formal process for STIX. So where is that line for TAXII ?
Clearly I think there are some things that we could easily identify as candidates for a more formal vetting process, those would be things like a solution for Query and TAXII Channels. But what else? Does a new
endpoint need this? If so, are there times when it would not?
Just some questions for the TC to think about. Also, as TAXII 2.1 does not currently have anything "new", just a bunch of changes / fixes from 2.0, I am not sure if this process is needed for 2.1, but I would be
happy to hear if people think differently.
One of the things that is not yet addressed is how does the TC decide to work on something new, either for STIX or TAXII, also, how does the TC decide if something should be included in a release of STIX or TAXII?
I wrote up a draft proposal for this, that we may also need to address at the same time as any process changes for TAXII.
Bret
From: John-Mark Gurney <
jmg@newcontext.com>
Sent: Friday, December 7, 2018 5:38:37 PM
To: Matt Pladna
Cc: Bret Jordan; Kelley, Sarah E.;
cti@lists.oasis-open.org Subject: Re: [cti] RE: [EXT] Re: [cti] TAXII definition of "Done"
Matt Pladna wrote this message on Fri, Dec 07, 2018 at 19:08 +0000:
> I support having TAXII meet some definition of Done Kelley.
I agree that TAXII needs to have the same definition of done as the rest
of the TC...
> Also, isn t this a process change vs a feature change to the spec? Or is the point that there should be a vote for this before it s decided?
>
> What would the # of people out of 200 be required to make this happen?
>
> From: <
cti@lists.oasis-open.org> on behalf of Bret Jordan <
Bret_Jordan@symantec.com>
> Date: Friday, December 7, 2018 at 14:04
> To: "Kelley, Sarah E." <
skelley@mitre.org>
> Cc: "cti@lists.oasis-open.org" <
cti@lists.oasis-open.org>
> Subject: Re: [cti] RE: [EXT] Re: [cti] TAXII definition of "Done"
>
> And as I said in my last email...
>
> While I fundamentally do not disagree, I have yet to see the TC push for this. By this philosophy we should have adopted the whole content process that I sent to the TC as a fully developed working draft several months ago.
>
>
> So unless there is a ground swell of the TC that is pushing for this, I would object simply out of principle. This is the same reason why tons of new things are not just added to TAXII, there is no demand for them and it is not up to me to just add them.
>
>
> So I object on principle. You have 4 people out of over 200 which is less than 2%. The TC has made it really clear as of late that talking about process is not something people want to do. And if we are going to try and talk about process then once again
I want to bring up the fully fleshed out Draft I submitted to the TC, which basically codifies what we have been doing behind the scenes.
>
> Bret
>
> Sent from my Commodore 128D
>
>
> PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050
>
> On Dec 7, 2018, at 10:08 AM, Kelley, Sarah E. <
skelley@mitre.org<mailto:
skelley@mitre.org>> wrote:
> Bret,
>
> As of last week, the people in support of having TAXII meet some definition of done was:
>
> Sarah Kelley
> Jason Keirstead
> John Wunder
> Allan Thompson
> and you:
>
> I am fine with that (since this is what I am doing behind the scenes anyways), but this would need to be taken to a ballot just like we did for STIX. It would need to be binding, not just a casual agreement.
> Since no one objected to the idea, and at least five people supported it, the goal was to move the ball further down the court and see if there was an appetite for taking this to a ballot and making it more official, and if so, to figure out when we might
want to do that.
>
> Thanks,
>
> Sarah Kelley
> Lead Cybersecurity Engineer, T8B2
> Defensive Operations
> The MITRE Corporation
> 703-983-6242
>
skelley@mitre.org< mailto:
skelley@mitre.org >
> <image003.jpg><https://clicktime.symantec.com/a/1/vncIdxSA3A4xal5shRajaDYU9DQpDQJrpTsBsIQYQT0=?d=ItDYMdKTUJlWiicY2nDNmyqFNII0tqf5Pi46Zp5KtCLRpKQg8SNHxmyrlL4EOK4HmyAfOTgA4Q9Q7Sk0NDox8-zxxIz01Og0Pzj2DbzvbUAHkcOYLaJmH8mwz26zgSn0vjNSh4H_Dyfm4QC2Mbdq9QoqElEWvgQbWogiLFm5Ib_KzA71P_uzGGFrFwNoWvmw9hzRDf4YkoRQIs6_i-PY_efoVGFnvmd0zR-hTYe9BuXxA2HnUrceX2a1Qo29Zz2a62Mt-1izHM1GYUOPNc8WYE7fX_U6X2mM8X1dxd9clDG5VBh16ZCTJrdh9EpEaLncI2gFzkxbpXKvjUblkAzmk8P26RPSjXyxAbiPSwJ1mpIQlKHc_Pw8b-fI5UD9qPMvQ23J_9yPEPVovKQkVokgNTYbqgxSrKx34F6wJ40Mf6NxiBTNWwnTkWdlNupAMVJ-GL5oEvDFzN_J7Fp8jML54Z3FnrTkgAK0csiCzz6KjH4ECxMIgpUVKsO_&u=http%3A%2F%2Fwww.mitre.org%2F>
>
> From: Bret Jordan <
Bret_Jordan@symantec.com<mailto:
Bret_Jordan@symantec.com>>
> Sent: Thursday, December 6, 2018 4:58 PM
> To: Kelley, Sarah E. <
skelley@mitre.org<mailto:
skelley@mitre.org>>
> Cc:
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>
> Subject: Re: [cti] RE: [EXT] Re: [cti] TAXII definition of "Done"
>
> In all things consensus based, there is the does anyone object and two, who supports and is driving this .
>
> It is generally not good form to do things by objection but rather first by demand and then by objection.
>
> While I fundamentally do not disagree, I have yet to see the TC push for this. By this philosophy we should have adopted the whole content process that I sent to the TC as a fully developed working draft several months ago.
>
> So unless there is a ground swell of the TC that is pushing for this, I would object simply out of principle. This is the same reason why tons of new things are not just added to TAXII, there is no demand for them and it is not up to me to just add them.
>
> Bret
> Sent from my Commodore 128D
>
>
>
> PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050
>
> On Dec 7, 2018, at 2:23 AM, Kelley, Sarah E. <
skelley@mitre.org<mailto:
skelley@mitre.org>> wrote:
> All,
>
> Having seen no objections to the idea of instituting a mandate of done for TAXII (in addition to STIX), I believe the next step would be to decide when we want to institute that policy. As with STIX, the best way to institute that new policy would be to
have a ballot on it, so we would need to decide when to open that ballot.
>
> In my understanding of the changes in the current WD that is open for ballot, the only new thing is the client user-agent. From my perspective, this seems like a relatively small change to hold up with the addition of this new process, however something
like TAXII query would make sense to have proven out in code and to pre-build interop tests for.
>
> What do the TC members think about when we should start the ball rolling on implementing this policy?
>
> Thanks,
>
> Sarah Kelley
> Lead Cybersecurity Engineer, T8B2
> Defensive Operations
> The MITRE Corporation
> 703-983-6242
>
skelley@mitre.org< mailto:
skelley@mitre.org >
> <image003.jpg><https://clicktime.symantec.com/a/1/lwDjxdQuulFcm7NdywUBnSZUR_C1zd9konyJfqfsEsY=?d=_VqGAtgxpuV0IihxvFhFPcPCaw5WlSzgXBXhLEDnfl0u1wBn3yk1h15q_FmLrtE2IKq8fovIPVep9N5rUqa9MYOg6MPIL_72-oESgkCFJAhRBrhiOGER2BCH092BrZADxR7BfgcE8XUGtnZ_aGIn9XdnAr7sMvxWBPPiefl46oMUf6n1dPEMqpwCA2PYJd7gdLuVwamjpR9GjZs3HxVD4GShJkU6ZoLo0NCwqlnqLUVGVVN_GhRrf4XYLSz3nXfwy6LjN2ttIw1NooiV5UeVKbFSD5u7d5UQaZYc7zxAGWLmx6cXKeK985MAXXVtMhXpAjGKUSYS2NlqSIjJ02egbSUDJOP8l84V9I0hjzF_YPTjiADe1PqA9B-eXthYSC_fqFGJvjziJHjrU_xFIBCfq2kEKtnWnh6lxlqxP7ScyML01NwaNvpF3pdn8JurxILn8fh1lwlFlvY5lLPfjUUE79nsZuDvqyFcVvCKBqW8KzL8nR1IN9E%3D&u=http%3A%2F%2Fwww.mitre.org%2F>
>
> From: Bret Jordan <
Bret_Jordan@symantec.com<mailto:
Bret_Jordan@symantec.com>>
> Sent: Tuesday, November 27, 2018 5:29 PM
> To: Allan Thomson <
athomson@lookingglasscyber.com<mailto:
athomson@lookingglasscyber.com>>; Wunder, John A. <
jwunder@mitre.org<mailto:
jwunder@mitre.org>>; Jason Keirstead <
Jason.Keirstead@ca.ibm.com<mailto:
Jason.Keirstead@ca.ibm.com>>; Kelley, Sarah E. <
skelley@mitre.org<mailto:
skelley@mitre.org>>
> Cc:
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>
> Subject: Re: [EXT] Re: [cti] TAXII definition of "Done"
>
>
> I am fine with that (since this is what I am doing behind the scenes anyways), but this would need to be taken to a ballot just like we did for STIX. It would need to be binding, not just a casual agreement.
>
>
>
> What I am doing right now is making sure every feature that gets added to TAXII is actually implemented in my libraries and test server (at some level). I am doing this to help prevent the problems we had with TAXII 2.0, where 20 minutes in to coding we
realized that, that design does not work in code. Some of the issues we have resolved in TAXII 2.1 have come about because of this code work that I and others have done and the plugfests we have held. I am a firm believer in "working code" and easy to implement
in code. I think those are two of the pillars to adoption.
>
>
>
> One of the differences we have in TAXII versus STIX though is, TAXII does not have features that are just conceptual models. STIX on the other hand can just be "modeled" and not implemented. This is why it was so important to have the "written in code" clause
for STIX.
>
>
>
> Bret
>
> ________________________________
> From:
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org> <
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>> on behalf of Allan Thomson <
athomson@lookingglasscyber.com<mailto:
athomson@lookingglasscyber.com>>
> Sent: Tuesday, November 27, 2018 2:26:44 PM
> To: Wunder, John A.; Jason Keirstead; Kelley, Sarah E.
> Cc:
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>
> Subject: [EXT] Re: [cti] TAXII definition of "Done"
>
>
> +1 to TAXII features starting to require the same level of doneness as STIX changes.
>
>
>
> Allan
>
>
>
> From: "
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>" <
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>> on behalf of "Wunder, John" <
jwunder@mitre.org<mailto:
jwunder@mitre.org>>
> Date: Tuesday, November 27, 2018 at 1:21 PM
> To: Jason Keirstead <
Jason.Keirstead@ca.ibm.com<mailto:
Jason.Keirstead@ca.ibm.com>>, "Kelley, Sarah E." <
skelley@mitre.org<mailto:
skelley@mitre.org>>
> Cc: "
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>" <
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>>
> Subject: Re: [cti] TAXII definition of "Done"
>
>
>
> Agreed, the same motivation for wanting to do this for STIX applies to TAXII. I d also keep in mind that requiring sponsors and interop text makes it so that you re not just evaluating technical feasibility (the implementation piece), you re also ensuring
that there s defined use cases and a real scenario where it can be used (a concern discussed on the call). It s way easier to say yes to something new than to say no, so it s important to have these checks in place to make sure we don t end up with something
overly broad again.
>
>
>
> John
>
>
>
> From: <
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>> on behalf of Jason Keirstead <
Jason.Keirstead@ca.ibm.com<mailto:
Jason.Keirstead@ca.ibm.com>>
> Date: Tuesday, November 27, 2018 at 4:15 PM
> To: "Kelley, Sarah E." <
skelley@mitre.org<mailto:
skelley@mitre.org>>
> Cc: "
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>" <
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>>
> Subject: Re: [cti] TAXII definition of "Done"
>
>
>
> I would also agree that TAXII features should also meet the STIX definition of "done" in order to be included in the spec.
>
> -
> Jason Keirstead
> Lead Architect - IBM Security Connect
>
www.ibm.com/security >
> "Things may come to those who wait, but only the things left by those who hustle." - Unknown
>
>
>
>
> From: "Kelley, Sarah E." <
skelley@mitre.org<mailto:
skelley@mitre.org>>
> To: "
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>" <
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>>
> Date: 11/27/2018 04:56 PM
> Subject: [cti] TAXII definition of "Done"
> Sent by: <
cti@lists.oasis-open.org<mailto:
cti@lists.oasis-open.org>>
>
> ________________________________
>
>
>
> All,
>
> As I mentioned on the working call today, we have imposed a very strict definition of Done for new features/objects in STIX, however, we have never agreed as a TC to impose the same rigorous standards to TAXII. Given the fact that some of the issues that
prompted us to implement this definition came about when people attempted to implement TAXII, it seems only logical to me that we would impose the same standards to both specifications.
>
> As a reminder, the definition of Done for STIX includes:
>
> 1. Written specification text
>
> 1. Proof of concept code from at least two different developers/companies
>
> 1. Corresponding Interop tests
>
>
> For some of the newer features in TAXII, namely TAXII query, it seems to make sense to me that it should be proved in code before we finalize it in the specification.
>
> I wanted to bring this topic to the list and see what other people thought about this.
>
> Thanks,
>
> Sarah Kelley
> Lead Cybersecurity Engineer, T8B2
> Defensive Operations
> The MITRE Corporation
> 703-983-6242
>
skelley@mitre.org< mailto:
skelley@mitre.org >
>
> [attachment "image003.jpg" deleted by Jason Keirstead/CanEast/IBM]
>
>
>
>
>
--
John-Mark