OASIS Cyber Threat Intelligence (CTI) TC

Hi Allan / All If that's how the documentation reads then we need to fix that.  The versioning working group had long discussions on the effects of immutability on the versioning process and of whether relationships should be specific to versions or transitive across all versions. After a couple of months of discussions we can to the conclusion that the only way to ensure that old relationships still worked was for each SDO  to stay describing the same thing as when it was first released. Small changes to update missing or incorrect information would be allowed through versioning, but major changes to the meaning of the object would be named, and would need to be done as a new object (under a new STIX id). The above restrictions mean that any previous relationships stay valid. The semantic meaning of the SDOs don't change, so the relationships will stay valid, as the changes will be minimal. This is such a key part of ensuring that versioning model runs smoothly in all parts of the model. The transitive nature of object relationships also greatly reduces the amount of network traffic sent as the SROs don't need to be updated - which Bret will like as he often brings bytes on the wire up :). If we haven't got enough documentation to explicitly state what I've highlighted above, then we really need to update the text to ensure that this is clear to implementers. Cheers Terry MacDonald Cosive On 20/06/2017 01:20, "Allan Thomson" < athomson@lookingglasscyber.com > wrote: Hi Rich – I think your point on versioning of a SDO materially is important to understand (by all implementers) and it’s important to note that there is nothing in the standard that precludes such changes.   Other than the object-id which remains immutable after object creation all other attributes are mutable for SDOs in the current specification.   I’m not sure we can change the specification to enforce anything else. Therefore, it’s possible that intel changes significantly from one version of an object to another.     Allan Thomson CTO +1-408-331-6646 LookingGlass Cyber Solutions   From: " cti@lists.oasis-open.org " < cti@lists.oasis-open.org > on behalf of "Piazza, Rich" < rpiazza@mitre.org > Date: Friday, June 16, 2017 at 6:10 AM To: John-Mark Gurney < jmg@newcontext.com > Cc: Bret Jordan < Bret_Jordan@symantec.com >, "Wunder, John" < jwunder@mitre.org >, Patrick Maroney < pmaroney@wapacklabs.com >, Jason Keirstead < Jason.Keirstead@ca.ibm.com >, " cti@lists.oasis-open.org " < cti@lists.oasis-open.org >, "Back, Greg" < gback@mitre.org >, " Nathan.Reller@jhuapl.edu " < Nathan.Reller@jhuapl.edu > Subject: Re: [cti] Re: [EXT] Re: [cti] [EXT] [cti] Location as a Top-Level SDO   John-Mark, you bring up an important point, which I, and maybe others, often forget – material changes should not be handled via versioning .   Therefore, no one should be changing their SDO, that your SDO is related to, in a way that makes the relationship invalid.    Your work flow below is the way this should he handled – revoke the SRO, and create a new one to the new SDO.   I’m not sure I understand your argument about the final flag – but based on the rest of your email – I think it is unnecessary (at least for this use case).   On 6/15/17, 6:53 PM, "John-Mark Gurney" < jmg@newcontext.com > wrote:       Piazza, Rich wrote this message on Wed, Jun 14, 2017 at 20:58 +0000:     > I don't think we should give up on the idea of reusing Locations so quickly.  Assuming we go with Locations as SDOs, it certainly is a problem if you reuse someone's Location and they change it from underneath you.   I was first thinking that there should be immutable SDOs - in other words, the unique USA Location SDO CAN'T be changed.   If we had a set of the common ones (defined in some library/repo somewhere)  then we could just use their ids.   Duplicates are allowed, but hopefully few people would need to create their own USA Location SDO.   I was thinking of an extra property (on all SDOs?)  - final.  If final is true for an SDO, then a new version couldn’t be created.           I'd like to point out we already have an "imutable" SDO.  The versioning     spec specifically calls out that if you make a material change to an     SDO, that you need to create a new one, and not reuse an existing one.         We might want to extend the text to say that if you created an SDO     and link it, but that the linked SDO was incorrect, say USA vs     USA Major Islands, that you need to create a new Location object,     and revoke the original relationship, and that changing the original     object is NOT the correct work flow.         The idea of a final flag is an interesting one, but what would the     handling of when a new final object is created?  Or a new one that     had the modified date before the other one?  This is just changing     the problem slightly w/o solving it.         Other solution is to simply say the Location objects cannot be versioned.         I cannot really think of a good reason/way to version/update a Location     object w/o materially altering it's meaning.         > Adding immutable objects to the spec might be a good idea in general, but I think a simpler way to handle this is just to "trust" that the library/repo contains objects that will not change.  In other words, locations created by a certain (well-known) identity (SDO-Immutable-Library) could be reused with little concern that they are going to change.  And if they DO change - maybe that is a feature - after all, all those Soviet-Union Location SDOs are no longer too useful...         --     John-Mark         

Terry, I agree with everything you are saying, but I believe my point still stands. The problem is how to express, in normative text, the definition of a material change (or, using your wording below, a large change ). What you may consider a material change in the meaning of an object,  I may not, and vice versa. At the end of the day, it is very subjective and will be open to interpretation. This will be decided in the end by various trust groups, information vendors, and ISAOs, all of whom I am sure will come up with policies to decide when there is a material change in their own objects vs. something that can be versioned. Sent from IBM Verse Terry MacDonald --- Re: [cti] Re: [EXT] Re: [cti] [EXT] [cti] Location as a Top-Level SDO --- From: "Terry MacDonald" <terry.macdonald@cosive.com> To: "Allan Thomson" <athomson@lookingglasscyber.com> Cc: "Rich Piazza" <rpiazza@mitre.org>, "Bret Jordan" <Bret_Jordan@symantec.com>, Nathan.Reller@jhuapl.edu, "John-Mark Gurney" <jmg@newcontext.com>, "Back, Greg" <gback@mitre.org>, cti@lists.oasis-open.org, "Wunder, John A." <jwunder@mitre.org>, "Jason Mr. Keirstead" <Jason.Keirstead@ca.ibm.com>, "Patrick Maroney" <pmaroney@wapacklabs.com> Date: Mon, Jun 19, 2017 5:09 PM Subject: Re: [cti] Re: [EXT] Re: [cti] [EXT] [cti] Location as a Top-Level SDO Hi Allan / All If that's how the documentation reads then we need to fix that.  The versioning working group had long discussions on the effects of immutability on the versioning process and of whether relationships should be specific to versions or transitive across all versions. After a couple of months of discussions we can to the conclusion that the only way to ensure that old relationships still worked was for each SDO  to stay describing the same thing as when it was first released. Small changes to update missing or incorrect information would be allowed through versioning, but major changes to the meaning of the object would be named, and would need to be done as a new object (under a new STIX id). The above restrictions mean that any previous relationships stay valid. The semantic meaning of the SDOs don't change, so the relationships will stay valid, as the changes will be minimal. This is such a key part of ensuring that versioning model runs smoothly in all parts of the model. The transitive nature of object relationships also greatly reduces the amount of network traffic sent as the SROs don't need to be updated - which Bret will like as he often brings bytes on the wire up :). If we haven't got enough documentation to explicitly state what I've highlighted above, then we really need to update the text to ensure that this is clear to implementers. Cheers Terry MacDonald Cosive On 20/06/2017 01:20, "Allan Thomson" < athomson@lookingglasscyber.com > wrote: Hi Rich – I think your point on versioning of a SDO materially is important to understand (by all implementers) and it’s important to note that there is nothing in the standard that precludes such changes.   Other than the object-id which remains immutable after object creation all other attributes are mutable for SDOs in the current specification.   I’m not sure we can change the specification to enforce anything else. Therefore, it’s possible that intel changes significantly from one version of an object to another.     Allan Thomson CTO +1-408-331-6646 LookingGlass Cyber Solutions   From: " cti@lists.oasis-open.org " < cti@lists.oasis-open.org > on behalf of "Piazza, Rich" < rpiazza@mitre.org > Date: Friday, June 16, 2017 at 6:10 AM To: John-Mark Gurney < jmg@newcontext.com > Cc: Bret Jordan < Bret_Jordan@symantec.com >, "Wunder, John" < jwunder@mitre.org >, Patrick Maroney < pmaroney@wapacklabs.com >, Jason Keirstead < Jason.Keirstead@ca.ibm.com >, " cti@lists.oasis-open.org " < cti@lists.oasis-open.org >, "Back, Greg" < gback@mitre.org >, " Nathan.Reller@jhuapl.edu " < Nathan.Reller@jhuapl.edu > Subject: Re: [cti] Re: [EXT] Re: [cti] [EXT] [cti] Location as a Top-Level SDO   John-Mark, you bring up an important point, which I, and maybe others, often forget – material changes should not be handled via versioning .   Therefore, no one should be changing their SDO, that your SDO is related to, in a way that makes the relationship invalid.    Your work flow below is the way this should he handled – revoke the SRO, and create a new one to the new SDO.   I’m not sure I understand your argument about the final flag – but based on the rest of your email – I think it is unnecessary (at least for this use case).   On 6/15/17, 6:53 PM, "John-Mark Gurney" < jmg@newcontext.com > wrote:       Piazza, Rich wrote this message on Wed, Jun 14, 2017 at 20:58 +0000:     > I don't think we should give up on the idea of reusing Locations so quickly.  Assuming we go with Locations as SDOs, it certainly is a problem if you reuse someone's Location and they change it from underneath you.   I was first thinking that there should be immutable SDOs - in other words, the unique USA Location SDO CAN'T be changed.   If we had a set of the common ones (defined in some library/repo somewhere)  then we could just use their ids.   Duplicates are allowed, but hopefully few people would need to create their own USA Location SDO.   I was thinking of an extra property (on all SDOs?)  - final.  If final is true for an SDO, then a new version couldn’t be created.           I'd like to point out we already have an "imutable" SDO.  The versioning     spec specifically calls out that if you make a material change to an     SDO, that you need to create a new one, and not reuse an existing one.         We might want to extend the text to say that if you created an SDO     and link it, but that the linked SDO was incorrect, say USA vs     USA Major Islands, that you need to create a new Location object,     and revoke the original relationship, and that changing the original     object is NOT the correct work flow.         The idea of a final flag is an interesting one, but what would the     handling of when a new final object is created?  Or a new one that     had the modified date before the other one?  This is just changing     the problem slightly w/o solving it.         Other solution is to simply say the Location objects cannot be versioned.         I cannot really think of a good reason/way to version/update a Location     object w/o materially altering it's meaning.         > Adding immutable objects to the spec might be a good idea in general, but I think a simpler way to handle this is just to "trust" that the library/repo contains objects that will not change.  In other words, locations created by a certain (well-known) identity (SDO-Immutable-Library) could be reused with little concern that they are going to change.  And if they DO change - maybe that is a feature - after all, all those Soviet-Union Location SDOs are no longer too useful...         --     John-Mark         

On 20.06.2017 03:27:30, Jason Keirstead wrote: > > The problem is how to express, in normative text, the definition of > a material change (or, using your wording below, a "large change"). > > What you may consider a material change in the meaning of an object, > I may not, and vice versa. At the end of the day, it is very > subjective and will be open to interpretation. > As Jason rightly points out, what constitutes a material change is subjective. Rather than attempting to quantify the subjective in normative text, let's add some examples illustrating the TC's intent to the STIX Implementer's Guide. -- Cheers, Trey ++--------------------------------------------------------------------------++ Director of Standards Development, New Context gpg fingerprint: 3918 9D7E 50F5 088F 823F 018A 831A 270A 6C4F C338 ++--------------------------------------------------------------------------++ -- "Good, Fast, Cheap: Pick any two (you can't have all three)." --RFC 1925 Attachment: signature.asc Description: Digital signature

Agree.
 

Allan Thomson
CTO
+1-408-331-6646

LookingGlass Cyber Solutions
 

From: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org> on behalf of Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Date: Monday, June 19, 2017 at 8:27 PM
To: Terry MacDonald <terry.macdonald@cosive.com>
Cc: Allan Thomson <athomson@lookingglasscyber.com>, Rich Piazza <rpiazza@mitre.org>, Bret Jordan <Bret_Jordan@symantec.com>, "Nathan.Reller@jhuapl.edu" <Nathan.Reller@jhuapl.edu>, John-Mark Gurney <jmg@newcontext.com>, "Back, Greg" <gback@mitre.org>,
"cti@lists.oasis-open.org" <cti@lists.oasis-open.org>, "Wunder, John" <jwunder@mitre.org>, Patrick Maroney <pmaroney@wapacklabs.com>
Subject: Re: [cti] Re: [EXT] Re: [cti] [EXT] [cti] Location as a Top-Level SDO


 

Terry, I agree with everything you are saying, but I believe my point still stands.

The problem is how to express, in normative text, the definition of a material change (or, using your wording below, a "large change").


What you may consider a material change in the meaning of an object,  I may not, and vice versa. At the end of the day, it is very subjective and will be open to interpretation.

This will be decided in the end by various trust groups, information vendors, and ISAOs, all of whom I am sure will come up with policies to decide when there is a material change in their own objects vs. something that can be versioned.



Sent from IBM Verse


Terry MacDonald --- Re: [cti] Re: [EXT] Re: [cti] [EXT] [cti] Location as a Top-Level SDO ---



 




From:


"Terry MacDonald" <terry.macdonald@cosive.com>




To:


"Allan Thomson" <athomson@lookingglasscyber.com>




Cc:


"Rich Piazza" <rpiazza@mitre.org>, "Bret Jordan" <Bret_Jordan@symantec.com>, Nathan.Reller@jhuapl.edu, "John-Mark Gurney" <jmg@newcontext.com>, "Back, Greg" <gback@mitre.org>, cti@lists.oasis-open.org, "Wunder,
John A." <jwunder@mitre.org>, "Jason Mr. Keirstead" <Jason.Keirstead@ca.ibm.com>, "Patrick Maroney" <pmaroney@wapacklabs.com>




Date:


Mon, Jun 19, 2017 5:09 PM




Subject:


Re: [cti] Re: [EXT] Re: [cti] [EXT] [cti] Location as a Top-Level SDO







 



Hi Allan / All

 


If that's how the documentation reads then we need to fix that. 


 


The versioning working group had long discussions on the effects of immutability on the versioning process and of whether relationships should be specific to versions or transitive across all versions. After a couple of months of discussions
we can to the conclusion that the only way to ensure that old relationships still worked was for each SDO  to stay describing the same thing as when it was first released. Small changes to update missing or incorrect information would be allowed through versioning,
but major changes to the meaning of the object would be named, and would need to be done as a new object (under a new STIX id).


 


The above restrictions mean that any previous relationships stay valid. The semantic meaning of the SDOs don't change, so the relationships will stay valid, as the changes will be minimal.


 


This is such a key part of ensuring that versioning model runs smoothly in all parts of the model. The transitive nature of object relationships also greatly reduces the amount of network traffic sent as the SROs don't need to be updated
- which Bret will like as he often brings bytes on the wire up :).


 

If we haven't got enough documentation to explicitly state what I've highlighted above, then we really need to update the text to ensure that this is clear to implementers.


 


Cheers


Terry MacDonald


Cosive

 

On 20/06/2017 01:20, "Allan Thomson" < athomson@lookingglasscyber.com > wrote:



Hi Rich – I think your point on versioning of a SDO materially is important to understand (by all implementers) and it’s important to note that there is nothing in the standard
that precludes such changes.
 
Other than the object-id which remains immutable after object creation all other attributes are mutable for SDOs in the current specification.
 
I’m not sure we can change the specification to enforce anything else. Therefore, it’s possible that intel changes significantly from one version of an object to another.

 
 

Allan Thomson
CTO
+1-408-331-6646

LookingGlass Cyber Solutions
 


From:
" cti@lists.oasis-open.org " < cti@lists.oasis-open.org > on behalf of "Piazza, Rich" < rpiazza@mitre.org >
Date: Friday, June 16, 2017 at 6:10 AM
To: John-Mark Gurney < jmg@newcontext.com >
Cc: Bret Jordan < Bret_Jordan@symantec.com >, "Wunder, John" < jwunder@mitre.org >, Patrick Maroney < pmaroney@wapacklabs.com >,
Jason Keirstead < Jason.Keirstead@ca.ibm.com >, " cti@lists.oasis-open.org " < cti@lists.oasis-open.org >,
"Back, Greg" < gback@mitre.org >, " Nathan.Reller@jhuapl.edu " < Nathan.Reller@jhuapl.edu >
Subject: Re: [cti] Re: [EXT] Re: [cti] [EXT] [cti] Location as a Top-Level SDO



 

John-Mark, you bring up an important point, which I, and maybe others, often forget –
material changes should not be handled via versioning .
 
Therefore, no one should be changing their SDO, that your SDO is related to, in a way that makes the relationship invalid. 

 
Your work flow below is the way this should he handled – revoke the SRO, and create a new one to the new SDO.
 
I’m not sure I understand your argument about the final flag – but based on the rest of your email – I think it is unnecessary (at least for this use case).

 
On 6/15/17, 6:53 PM, "John-Mark Gurney" < jmg@newcontext.com > wrote:
 
    Piazza, Rich wrote this message on Wed, Jun 14, 2017 at 20:58 +0000:
    > I don't think we should give up on the idea of reusing Locations so quickly.  Assuming we go with Locations as SDOs, it certainly is a problem if you reuse someone's Location and they change it from underneath
you.   I was first thinking that there should be immutable SDOs - in other words, the unique USA Location SDO CAN'T be changed.   If we had a set of the common ones (defined in some library/repo somewhere)  then we could just use their ids.   Duplicates are
allowed, but hopefully few people would need to create their own USA Location SDO.   I was thinking of an extra property (on all SDOs?)  - final.  If final is true for an SDO, then a new version couldn’t be created. 

    
    I'd like to point out we already have an "imutable" SDO.  The versioning
    spec specifically calls out that if you make a material change to an
    SDO, that you need to create a new one, and not reuse an existing one.
   
    We might want to extend the text to say that if you created an SDO
    and link it, but that the linked SDO was incorrect, say USA vs
    USA Major Islands, that you need to create a new Location object,
    and revoke the original relationship, and that changing the original
    object is NOT the correct work flow.
   
    The idea of a final flag is an interesting one, but what would the
    handling of when a new final object is created?  Or a new one that
    had the modified date before the other one?  This is just changing
    the problem slightly w/o solving it.
   
    Other solution is to simply say the Location objects cannot be versioned.
   
    I cannot really think of a good reason/way to version/update a Location
    object w/o materially altering it's meaning.
   
    > Adding immutable objects to the spec might be a good idea in general, but I think a simpler way to handle this is just to "trust" that the library/repo contains objects that will not change.  In other words,
locations created by a certain (well-known) identity (SDO-Immutable-Library) could be reused with little concern that they are going to change.  And if they DO change - maybe that is a feature - after all, all those Soviet-Union Location SDOs are no longer
too useful...
   
    --
    John-Mark