CTI STIX Subcommittee

Did this get lost in the shuffle last week? I didn't see any discussion on this. ----- Forwarded message from John-Mark Gurney <jmg@newcontext.com> ----- Date: Mon, 22 Aug 2016 17:08:44 -0700 From: John-Mark Gurney <jmg@newcontext.com> To: "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> Subject: Observed Data comments. Hello, Sorry for not getting this in sooner, but I have questions/comments about the Observed Data object. There is nothing in the spec that requires first_observed to be equal to last_observed when number_observed is 1. How is a tool who receives such an object to handle this? In the case of when you don't know exactly when the observed data was recorded, the _presision property added to first_observed can convey this information correctly, and most acurately. P.S. IMO, we should also make the number_observed default to 1, and make it optional. By setting a resonable default, we can reduce the size of objects in common cases. -- John-Mark ----- End forwarded message ----- -- John-Mark

How should we address this?  Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot.  Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On Aug 30, 2016, at 12:25, John-Mark Gurney < jmg@newcontext.com > wrote: Did this get lost in the shuffle last week?  I didn't see any discussion on this. ----- Forwarded message from John-Mark Gurney < jmg@newcontext.com > ----- Date: Mon, 22 Aug 2016 17:08:44 -0700 From: John-Mark Gurney < jmg@newcontext.com > To: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: Observed Data comments. Hello, Sorry for not getting this in sooner, but I have questions/comments about the Observed Data object. There is nothing in the spec that requires first_observed to be equal to last_observed when number_observed is 1. How is a tool who receives such an object to handle this? In the case of when you don't know exactly when the observed data was recorded, the _presision property added to first_observed can convey this information correctly, and most acurately. P.S. IMO, we should also make the number_observed default to 1, and make it optional.  By setting a resonable default, we can reduce the size of objects in common cases. -- John-Mark ----- End forwarded message ----- -- John-Mark --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php Attachment: signature.asc Description: Message signed with OpenPGP using GPGMail

When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional
conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor.
 
On the issues themselves:
-          
The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so
a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one.
-          
I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this.
John
 

From:
<cti-stix@lists.oasis-open.org> on behalf of Bret Jordan <bret.jordan@bluecoat.com>
Date: Tuesday, August 30, 2016 at 4:10 PM
To: John-Mark Gurney <jmg@newcontext.com>
Cc: "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]


 



How should we address this?  Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot.  Keep in mind we can do as many Committee Specification Draft releases we want before we
go to a Committee Specification with Public Review.







 


Thanks,


 


Bret



 


 


 



Bret Jordan CISSP

Director of Security Architecture and Standards Office of the CTO


Blue Coat Systems



PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050


"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 









 



On Aug 30, 2016, at 12:25, John-Mark Gurney < jmg@newcontext.com > wrote:

 


Did this get lost in the shuffle last week?  I didn't see any discussion
on this.

----- Forwarded message from John-Mark Gurney < jmg@newcontext.com > -----

Date: Mon, 22 Aug 2016 17:08:44 -0700
From: John-Mark Gurney < jmg@newcontext.com >
To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
Subject: Observed Data comments.

Hello,

Sorry for not getting this in sooner, but I have questions/comments
about the Observed Data object.

There is nothing in the spec that requires first_observed to be equal
to last_observed when number_observed is 1.

How is a tool who receives such an object to handle this?

In the case of when you don't know exactly when the observed data
was recorded, the _presision property added to first_observed can
convey this information correctly, and most acurately.

P.S. IMO, we should also make the number_observed default to 1, and
make it optional.  By setting a resonable default, we can reduce
the size of objects in common cases.

--
John-Mark

----- End forwarded message -----

--
John-Mark

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php





 

> - I would be fine making count optional and defaulting to 1. I’m also fine > as-is, would like to hear from more people about whether we should change > this. I agree with this. In the past I've heard people be critical of "optional with default value" approaches, but I think we should use them as much as possible. It reduces the space needed to transmit, with no loss in information.

For network-connection though, there are time fields in CybOX land.  So how do those relate to the time stamps in the Observed Data object.???  Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On Aug 30, 2016, at 14:28, Wunder, John A. < jwunder@mitre.org > wrote: When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor.   On the issues themselves: -             The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. -             I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. John   From:   < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date:   Tuesday, August 30, 2016 at 4:10 PM To:   John-Mark Gurney < jmg@newcontext.com > Cc:   cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject:   Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   How should we address this?  Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot.  Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review.   Thanks,   Bret       Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.     On Aug 30, 2016, at 12:25, John-Mark Gurney < jmg@newcontext.com > wrote:   Did this get lost in the shuffle last week?  I didn't see any discussion on this. ----- Forwarded message from John-Mark Gurney < jmg@newcontext.com > ----- Date: Mon, 22 Aug 2016 17:08:44 -0700 From: John-Mark Gurney < jmg@newcontext.com > To: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: Observed Data comments. Hello, Sorry for not getting this in sooner, but I have questions/comments about the Observed Data object. There is nothing in the spec that requires first_observed to be equal to last_observed when number_observed is 1. How is a tool who receives such an object to handle this? In the case of when you don't know exactly when the observed data was recorded, the _presision property added to first_observed can convey this information correctly, and most acurately. P.S. IMO, we should also make the number_observed default to 1, and make it optional.  By setting a resonable default, we can reduce the size of objects in common cases. --   John-Mark ----- End forwarded message ----- --   John-Mark --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that   generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php   Attachment: signature.asc Description: Message signed with OpenPGP using GPGMail

The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed
and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection.
 
How do you think it should work?
 

From:
Bret Jordan <bret.jordan@bluecoat.com>
Date: Tuesday, August 30, 2016 at 4:56 PM
To: "Wunder, John A." <jwunder@mitre.org>
Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]


 



For network-connection though, there are time fields in CybOX land.  So how do those relate to the time stamps in the Observed Data object.??? 







 


Thanks,


 


Bret



 


 


 



Bret Jordan CISSP

Director of Security Architecture and Standards Office of the CTO


Blue Coat Systems



PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050


"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 









 



On Aug 30, 2016, at 14:28, Wunder, John A. < jwunder@mitre.org > wrote:

 


When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway
since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor.


 


On the issues themselves:


-             The
reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have
a requirement that they be the same when the count is one.


-             I
would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this.


John


 



From:   < cti-stix@lists.oasis-open.org >
on behalf of Bret Jordan < bret.jordan@bluecoat.com >
Date:   Tuesday, August 30, 2016 at 4:10 PM
To:   John-Mark Gurney < jmg@newcontext.com >
Cc:   " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
Subject:   Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]




 





How should we address this?  Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot.  Keep in mind we can do as many Committee Specification Draft releases
we want before we go to a Committee Specification with Public Review.








 



Thanks,




 




Bret





 




 




 





Bret Jordan CISSP



Director of Security Architecture and Standards Office of the CTO




Blue Coat Systems





PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050




"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 











 





On Aug 30, 2016, at 12:25, John-Mark Gurney < jmg@newcontext.com > wrote:



 



Did this get lost in the shuffle last week?  I didn't see any discussion
on this.

----- Forwarded message from John-Mark Gurney < jmg@newcontext.com > -----

Date: Mon, 22 Aug 2016 17:08:44 -0700
From: John-Mark Gurney < jmg@newcontext.com >
To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
Subject: Observed Data comments.

Hello,

Sorry for not getting this in sooner, but I have questions/comments
about the Observed Data object.

There is nothing in the spec that requires first_observed to be equal
to last_observed when number_observed is 1.

How is a tool who receives such an object to handle this?

In the case of when you don't know exactly when the observed data
was recorded, the _presision property added to first_observed can
convey this information correctly, and most acurately.

P.S. IMO, we should also make the number_observed default to 1, and
make it optional.  By setting a resonable default, we can reduce
the size of objects in common cases.

--  
John-Mark

----- End forwarded message -----

--  
John-Mark

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that  
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php





 






 

Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:12 +0000: > The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection. > > How do you think it should work? If a sensor delays sending out the object until connection closes, it means that this data is not available to decide if action should be taken. This leaves a huge gap in coverage. IMO, if Observed Data isn't a single point in time, it will become more complicated to analyze the data. Representing connection open/closed can be easily recorded w/ two objects recording the state, the first one w/ state of CONNECTED, and a second one w/ the state CLOSED. As the four-tuple is required to be unique, this won't cause any confusion between connections unless objects are lost. I used -connected and -closed because CybOX says: Specifies the protocols used in the network connection, along with their corresponding state. But provides no examples of how to convey the state information, and the TCP Extension does not have it, though it could possibly done w/ flags (via SYN/FIN). { "type":"cybox-container", "spec_version":"3.0", "objects":{ "0":{ "type":"ipv4-address-object", "value":"1.2.3.4" }, "1":{ "type":"ipv4-address-object", "value":"2.3.4.5" }, "2":{ "type":"network-connection-object", "src_ref":"0", "dst_ref":"1", "protocols":[ "ipv4", "tcp-connected" ] } } } { "type":"cybox-container", "spec_version":"3.0", "objects":{ "0":{ "type":"ipv4-address-object", "value":"1.2.3.4" }, "1":{ "type":"ipv4-address-object", "value":"2.3.4.5" }, "2":{ "type":"network-connection-object", "src_ref":"0", "dst_ref":"1", "protocols":[ "ipv4", "tcp-closed" ] } } } > From: Bret Jordan <bret.jordan@bluecoat.com> > Date: Tuesday, August 30, 2016 at 4:56 PM > To: "Wunder, John A." <jwunder@mitre.org> > Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> > Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] > > For network-connection though, there are time fields in CybOX land. So how do those relate to the time stamps in the Observed Data object.??? > > Thanks, > > Bret > > > > Bret Jordan CISSP > Director of Security Architecture and Standards Office of the CTO > Blue Coat Systems > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > On Aug 30, 2016, at 14:28, Wunder, John A. <jwunder@mitre.org< mailto:jwunder@mitre.org >> wrote: > > When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor. > > On the issues themselves: > - The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. > - I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. > John > > From: <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> on behalf of Bret Jordan <bret.jordan@bluecoat.com< mailto:bret.jordan@bluecoat.com >> > Date: Tuesday, August 30, 2016 at 4:10 PM > To: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > Cc: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > Subject: Re: [cti-stix] [jmg@newcontext.com< mailto:jmg@newcontext.com >: Observed Data comments.] > > How should we address this? Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot. Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review. > > Thanks, > > Bret > > > > Bret Jordan CISSP > Director of Security Architecture and Standards Office of the CTO > Blue Coat Systems > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > On Aug 30, 2016, at 12:25, John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> wrote: > > Did this get lost in the shuffle last week? I didn't see any discussion > on this. > > ----- Forwarded message from John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> ----- > > Date: Mon, 22 Aug 2016 17:08:44 -0700 > From: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > To: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > Subject: Observed Data comments. > > Hello, > > Sorry for not getting this in sooner, but I have questions/comments > about the Observed Data object. > > There is nothing in the spec that requires first_observed to be equal > to last_observed when number_observed is 1. > > How is a tool who receives such an object to handle this? > > In the case of when you don't know exactly when the observed data > was recorded, the _presision property added to first_observed can > convey this information correctly, and most acurately. > > P.S. IMO, we should also make the number_observed default to 1, and > make it optional. By setting a resonable default, we can reduce > the size of objects in common cases. > > -- > John-Mark > > ----- End forwarded message ----- -- John-Mark

I guess my point of view is that some observations just by nature inherently occur over a period of time and we need to deal with it. Yes, you can send a connection opened event and a connection closed event but in reality for matching and analysis you’re going to need to operate over that entire time window. Same thing even for file create (which can take time) or more complex actions. Some things occur over a time window and IMO we need to capture that time window in Observed Data. John On 8/30/16, 5:27 PM, "John-Mark Gurney" <jmg@newcontext.com> wrote: Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:12 +0000: > The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection. > > How do you think it should work? If a sensor delays sending out the object until connection closes, it means that this data is not available to decide if action should be taken. This leaves a huge gap in coverage. IMO, if Observed Data isn't a single point in time, it will become more complicated to analyze the data. Representing connection open/closed can be easily recorded w/ two objects recording the state, the first one w/ state of CONNECTED, and a second one w/ the state CLOSED. As the four-tuple is required to be unique, this won't cause any confusion between connections unless objects are lost. I used -connected and -closed because CybOX says: Specifies the protocols used in the network connection, along with their corresponding state. But provides no examples of how to convey the state information, and the TCP Extension does not have it, though it could possibly done w/ flags (via SYN/FIN). { "type":"cybox-container", "spec_version":"3.0", "objects":{ "0":{ "type":"ipv4-address-object", "value":"1.2.3.4" }, "1":{ "type":"ipv4-address-object", "value":"2.3.4.5" }, "2":{ "type":"network-connection-object", "src_ref":"0", "dst_ref":"1", "protocols":[ "ipv4", "tcp-connected" ] } } } { "type":"cybox-container", "spec_version":"3.0", "objects":{ "0":{ "type":"ipv4-address-object", "value":"1.2.3.4" }, "1":{ "type":"ipv4-address-object", "value":"2.3.4.5" }, "2":{ "type":"network-connection-object", "src_ref":"0", "dst_ref":"1", "protocols":[ "ipv4", "tcp-closed" ] } } } > From: Bret Jordan <bret.jordan@bluecoat.com> > Date: Tuesday, August 30, 2016 at 4:56 PM > To: "Wunder, John A." <jwunder@mitre.org> > Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> > Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] > > For network-connection though, there are time fields in CybOX land. So how do those relate to the time stamps in the Observed Data object.??? > > Thanks, > > Bret > > > > Bret Jordan CISSP > Director of Security Architecture and Standards Office of the CTO > Blue Coat Systems > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > On Aug 30, 2016, at 14:28, Wunder, John A. <jwunder@mitre.org< mailto:jwunder@mitre.org >> wrote: > > When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor. > > On the issues themselves: > - The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. > - I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. > John > > From: <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> on behalf of Bret Jordan <bret.jordan@bluecoat.com< mailto:bret.jordan@bluecoat.com >> > Date: Tuesday, August 30, 2016 at 4:10 PM > To: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > Cc: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > Subject: Re: [cti-stix] [jmg@newcontext.com< mailto:jmg@newcontext.com >: Observed Data comments.] > > How should we address this? Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot. Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review. > > Thanks, > > Bret > > > > Bret Jordan CISSP > Director of Security Architecture and Standards Office of the CTO > Blue Coat Systems > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > On Aug 30, 2016, at 12:25, John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> wrote: > > Did this get lost in the shuffle last week? I didn't see any discussion > on this. > > ----- Forwarded message from John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> ----- > > Date: Mon, 22 Aug 2016 17:08:44 -0700 > From: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > To: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > Subject: Observed Data comments. > > Hello, > > Sorry for not getting this in sooner, but I have questions/comments > about the Observed Data object. > > There is nothing in the spec that requires first_observed to be equal > to last_observed when number_observed is 1. > > How is a tool who receives such an object to handle this? > > In the case of when you don't know exactly when the observed data > was recorded, the _presision property added to first_observed can > convey this information correctly, and most acurately. > > P.S. IMO, we should also make the number_observed default to 1, and > make it optional. By setting a resonable default, we can reduce > the size of objects in common cases. > > -- > John-Mark > > ----- End forwarded message ----- -- John-Mark

Sorry I had a paragraph here that I removed in my rush to get home: sending a network connection opening, as a discrete event, would make sense to me and is supported now (make the timestamps the same). I just don't think we should make discrete start/stop or open/close events the ONLY way to do it by requiring they be the same for single counts. From: Wunder, John A. < jwunder@mitre.org > Date: Tuesday, Aug 30, 2016, 5:38 PM To: John-Mark Gurney < jmg@newcontext.com > Cc: Jordan, Bret < bret.jordan@bluecoat.com >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] I guess my point of view is that some observations just by nature inherently occur over a period of time and we need to deal with it. Yes, you can send a connection opened event and a connection closed event but in reality for matching and analysis you’re going to need to operate over that entire time window. Same thing even for file create (which can take time) or more complex actions. Some things occur over a time window and IMO we need to capture that time window in Observed Data. John On 8/30/16, 5:27 PM, "John-Mark Gurney" <jmg@newcontext.com> wrote:     Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:12 +0000:     > The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection.     >     > How do you think it should work?         If a sensor delays sending out the object until connection closes, it     means that this data is not available to decide if action should be     taken.  This leaves a huge gap in coverage.         IMO, if Observed Data isn't a single point in time, it will become more     complicated to analyze the data.         Representing connection open/closed can be easily recorded w/ two     objects recording the state, the first one w/ state of CONNECTED,     and a second one w/ the state CLOSED.  As the four-tuple is required     to be unique, this won't cause any confusion between connections unless     objects are lost.         I used -connected and -closed because CybOX says:     Specifies the protocols used in the network connection, along with their corresponding state.         But provides no examples of how to convey the state information, and     the TCP Extension does not have it, though it could possibly done     w/ flags (via SYN/FIN).         {       "type":"cybox-container",       "spec_version":"3.0",       "objects":{         "0":{           "type":"ipv4-address-object",           "value":"1.2.3.4"         },         "1":{           "type":"ipv4-address-object",           "value":"2.3.4.5"         },         "2":{           "type":"network-connection-object",           "src_ref":"0",           "dst_ref":"1",           "protocols":[ "ipv4", "tcp-connected" ]         }       }     }         {       "type":"cybox-container",       "spec_version":"3.0",       "objects":{         "0":{           "type":"ipv4-address-object",           "value":"1.2.3.4"         },         "1":{           "type":"ipv4-address-object",           "value":"2.3.4.5"         },         "2":{           "type":"network-connection-object",           "src_ref":"0",           "dst_ref":"1",           "protocols":[ "ipv4", "tcp-closed" ]         }       }     }         > From: Bret Jordan <bret.jordan@bluecoat.com>     > Date: Tuesday, August 30, 2016 at 4:56 PM     > To: "Wunder, John A." <jwunder@mitre.org>     > Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>     > Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]     >     > For network-connection though, there are time fields in CybOX land.  So how do those relate to the time stamps in the Observed Data object.???     >     > Thanks,     >     > Bret     >     >     >     > Bret Jordan CISSP     > Director of Security Architecture and Standards Office of the CTO     > Blue Coat Systems     > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050     > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."     >     > On Aug 30, 2016, at 14:28, Wunder, John A. <jwunder@mitre.org<mailto:jwunder@mitre.org>> wrote:     >     > When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor.     >     > On the issues themselves:     > -          The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one.     > -          I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this.     > John     >     > From: <cti-stix@lists.oasis-open.org<mailto:cti-stix@lists.oasis-open.org>> on behalf of Bret Jordan <bret.jordan@bluecoat.com<mailto:bret.jordan@bluecoat.com>>     > Date: Tuesday, August 30, 2016 at 4:10 PM     > To: John-Mark Gurney <jmg@newcontext.com<mailto:jmg@newcontext.com>>     > Cc: "cti-stix@lists.oasis-open.org<mailto:cti-stix@lists.oasis-open.org>" <cti-stix@lists.oasis-open.org<mailto:cti-stix@lists.oasis-open.org>>     > Subject: Re: [cti-stix] [jmg@newcontext.com<mailto:jmg@newcontext.com>: Observed Data comments.]     >     > How should we address this?  Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot.  Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review.     >     > Thanks,     >     > Bret     >     >     >     > Bret Jordan CISSP     > Director of Security Architecture and Standards Office of the CTO     > Blue Coat Systems     > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050     > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."     >     > On Aug 30, 2016, at 12:25, John-Mark Gurney <jmg@newcontext.com<mailto:jmg@newcontext.com>> wrote:     >     > Did this get lost in the shuffle last week?  I didn't see any discussion     > on this.     >     > ----- Forwarded message from John-Mark Gurney <jmg@newcontext.com<mailto:jmg@newcontext.com>> -----     >     > Date: Mon, 22 Aug 2016 17:08:44 -0700     > From: John-Mark Gurney <jmg@newcontext.com<mailto:jmg@newcontext.com>>     > To: "cti-stix@lists.oasis-open.org<mailto:cti-stix@lists.oasis-open.org>" <cti-stix@lists.oasis-open.org<mailto:cti-stix@lists.oasis-open.org>>     > Subject: Observed Data comments.     >     > Hello,     >     > Sorry for not getting this in sooner, but I have questions/comments     > about the Observed Data object.     >     > There is nothing in the spec that requires first_observed to be equal     > to last_observed when number_observed is 1.     >     > How is a tool who receives such an object to handle this?     >     > In the case of when you don't know exactly when the observed data     > was recorded, the _presision property added to first_observed can     > convey this information correctly, and most acurately.     >     > P.S. IMO, we should also make the number_observed default to 1, and     > make it optional.  By setting a resonable default, we can reduce     > the size of objects in common cases.     >     > --     > John-Mark     >     > ----- End forwarded message -----         --     John-Mark    

Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:46 +0000: > Sorry I had a paragraph here that I removed in my rush to get home: sending a network connection opening, as a discrete event, would make sense to me and is supported now (make the timestamps the same). I just don't think we should make discrete start/stop or open/close events the ONLY way to do it by requiring they be the same for single counts. If we end up w/ multiple ways to do the same thing, then writing a pattern will not work genericly. We'll end up w/ a pattern that only works when it used w/ data from sensor A, but not B because of the different formats for the same information. John-Mark > From: Wunder, John A. <jwunder@mitre.org< mailto:jwunder@mitre.org >> > Date: Tuesday, Aug 30, 2016, 5:38 PM > To: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > Cc: Jordan, Bret <bret.jordan@bluecoat.com< mailto:bret.jordan@bluecoat.com >>, cti-stix@lists.oasis-open.org <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] > > I guess my point of view is that some observations just by nature inherently occur over a period of time and we need to deal with it. Yes, you can send a connection opened event and a connection closed event but in reality for matching and analysis you’re going to need to operate over that entire time window. Same thing even for file create (which can take time) or more complex actions. Some things occur over a time window and IMO we need to capture that time window in Observed Data. > > John > > > On 8/30/16, 5:27 PM, "John-Mark Gurney" <jmg@newcontext.com> wrote: > > Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:12 +0000: > > The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection. > > > > How do you think it should work? > > If a sensor delays sending out the object until connection closes, it > means that this data is not available to decide if action should be > taken. This leaves a huge gap in coverage. > > IMO, if Observed Data isn't a single point in time, it will become more > complicated to analyze the data. > > Representing connection open/closed can be easily recorded w/ two > objects recording the state, the first one w/ state of CONNECTED, > and a second one w/ the state CLOSED. As the four-tuple is required > to be unique, this won't cause any confusion between connections unless > objects are lost. > > I used -connected and -closed because CybOX says: > Specifies the protocols used in the network connection, along with their corresponding state. > > But provides no examples of how to convey the state information, and > the TCP Extension does not have it, though it could possibly done > w/ flags (via SYN/FIN). > > { > "type":"cybox-container", > "spec_version":"3.0", > "objects":{ > "0":{ > "type":"ipv4-address-object", > "value":"1.2.3.4" > }, > "1":{ > "type":"ipv4-address-object", > "value":"2.3.4.5" > }, > "2":{ > "type":"network-connection-object", > "src_ref":"0", > "dst_ref":"1", > "protocols":[ "ipv4", "tcp-connected" ] > } > } > } > > { > "type":"cybox-container", > "spec_version":"3.0", > "objects":{ > "0":{ > "type":"ipv4-address-object", > "value":"1.2.3.4" > }, > "1":{ > "type":"ipv4-address-object", > "value":"2.3.4.5" > }, > "2":{ > "type":"network-connection-object", > "src_ref":"0", > "dst_ref":"1", > "protocols":[ "ipv4", "tcp-closed" ] > } > } > } > > > From: Bret Jordan <bret.jordan@bluecoat.com> > > Date: Tuesday, August 30, 2016 at 4:56 PM > > To: "Wunder, John A." <jwunder@mitre.org> > > Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> > > Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] > > > > For network-connection though, there are time fields in CybOX land. So how do those relate to the time stamps in the Observed Data object.??? > > > > Thanks, > > > > Bret > > > > > > > > Bret Jordan CISSP > > Director of Security Architecture and Standards Office of the CTO > > Blue Coat Systems > > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > > > On Aug 30, 2016, at 14:28, Wunder, John A. <jwunder@mitre.org< mailto:jwunder@mitre.org >> wrote: > > > > When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor. > > > > On the issues themselves: > > - The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. > > - I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. > > John > > > > From: <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> on behalf of Bret Jordan <bret.jordan@bluecoat.com< mailto:bret.jordan@bluecoat.com >> > > Date: Tuesday, August 30, 2016 at 4:10 PM > > To: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > > Cc: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > > Subject: Re: [cti-stix] [jmg@newcontext.com< mailto:jmg@newcontext.com >: Observed Data comments.] > > > > How should we address this? Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot. Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review. > > > > Thanks, > > > > Bret > > > > > > > > Bret Jordan CISSP > > Director of Security Architecture and Standards Office of the CTO > > Blue Coat Systems > > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > > > On Aug 30, 2016, at 12:25, John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> wrote: > > > > Did this get lost in the shuffle last week? I didn't see any discussion > > on this. > > > > ----- Forwarded message from John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> ----- > > > > Date: Mon, 22 Aug 2016 17:08:44 -0700 > > From: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > > To: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > > Subject: Observed Data comments. > > > > Hello, > > > > Sorry for not getting this in sooner, but I have questions/comments > > about the Observed Data object. > > > > There is nothing in the spec that requires first_observed to be equal > > to last_observed when number_observed is 1. > > > > How is a tool who receives such an object to handle this? > > > > In the case of when you don't know exactly when the observed data > > was recorded, the _presision property added to first_observed can > > convey this information correctly, and most acurately. > > > > P.S. IMO, we should also make the number_observed default to 1, and > > make it optional. By setting a resonable default, we can reduce > > the size of objects in common cases. > > > > -- > > John-Mark > > > > ----- End forwarded message ----- > > -- > John-Mark > > > -- John-Mark

So here’s my perspective. I’m not super involved in patterning so take those portions with a grain of salt, and to be honest I don’t feel super strongly about any of this. I just want to explain why I think what we have works semantically. - Having each network connection require two Observed Data instances to represent the start and the stop seems kind of crazy to me. I’m not sure tools would get that right and having double the messages is going to be an issue. - Matching is rarely going to occur against literal Observed Data instances in actual products (IMO). STIX Observed Data is a serialization format. Matching can be written as if it’s against Observed Data if that helps explain how it works, but the raw data is the raw data. Making Observed Data have different semantics to make matching easier seems to be the wrong way to go about this to me. Patterning needs an approach to matching network connections (and other events) that occur over time ranges and figuring out some simple approach for that would absolutely make sense, but that doesn’t need to change the semantics of observed data. Maybe you can have some normative text in the patterning specification that describes how temporal verbs match against time windows. - I don’t think what I said was multiple ways to do the same thing. Observed Data captures the metadata about an observation, including the time that the observation was captured. If I observe a network connection that just opened then my first/last observed will be the same because I observed that network connection over that time. If I observe it a minute later when it’s still open, then my first observed is when I first started capturing the data and my last observed is when I last started capturing it. If I observe it when it’s closed, the first observed is when you first started capturing the data and the last observed is when you finished capturing that data, aka when the connection closed. These are all just observations of the same network connection, just as you can have multiple observations (observed data) of the same file. If you want to write a pattern to match that network connection you need to assume that the connection occurs over a time window, because it does. Some sensors might output multiple observations of that connection, others might just report one when it’s over, but the raw data you’re matching against is the same either way. John On 8/30/16, 6:05 PM, "cti-stix@lists.oasis-open.org on behalf of John-Mark Gurney" <cti-stix@lists.oasis-open.org on behalf of jmg@newcontext.com> wrote: Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:46 +0000: > Sorry I had a paragraph here that I removed in my rush to get home: sending a network connection opening, as a discrete event, would make sense to me and is supported now (make the timestamps the same). I just don't think we should make discrete start/stop or open/close events the ONLY way to do it by requiring they be the same for single counts. If we end up w/ multiple ways to do the same thing, then writing a pattern will not work genericly. We'll end up w/ a pattern that only works when it used w/ data from sensor A, but not B because of the different formats for the same information. John-Mark > From: Wunder, John A. <jwunder@mitre.org< mailto:jwunder@mitre.org >> > Date: Tuesday, Aug 30, 2016, 5:38 PM > To: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > Cc: Jordan, Bret <bret.jordan@bluecoat.com< mailto:bret.jordan@bluecoat.com >>, cti-stix@lists.oasis-open.org <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] > > I guess my point of view is that some observations just by nature inherently occur over a period of time and we need to deal with it. Yes, you can send a connection opened event and a connection closed event but in reality for matching and analysis you’re going to need to operate over that entire time window. Same thing even for file create (which can take time) or more complex actions. Some things occur over a time window and IMO we need to capture that time window in Observed Data. > > John > > > On 8/30/16, 5:27 PM, "John-Mark Gurney" <jmg@newcontext.com> wrote: > > Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:12 +0000: > > The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection. > > > > How do you think it should work? > > If a sensor delays sending out the object until connection closes, it > means that this data is not available to decide if action should be > taken. This leaves a huge gap in coverage. > > IMO, if Observed Data isn't a single point in time, it will become more > complicated to analyze the data. > > Representing connection open/closed can be easily recorded w/ two > objects recording the state, the first one w/ state of CONNECTED, > and a second one w/ the state CLOSED. As the four-tuple is required > to be unique, this won't cause any confusion between connections unless > objects are lost. > > I used -connected and -closed because CybOX says: > Specifies the protocols used in the network connection, along with their corresponding state. > > But provides no examples of how to convey the state information, and > the TCP Extension does not have it, though it could possibly done > w/ flags (via SYN/FIN). > > { > "type":"cybox-container", > "spec_version":"3.0", > "objects":{ > "0":{ > "type":"ipv4-address-object", > "value":"1.2.3.4" > }, > "1":{ > "type":"ipv4-address-object", > "value":"2.3.4.5" > }, > "2":{ > "type":"network-connection-object", > "src_ref":"0", > "dst_ref":"1", > "protocols":[ "ipv4", "tcp-connected" ] > } > } > } > > { > "type":"cybox-container", > "spec_version":"3.0", > "objects":{ > "0":{ > "type":"ipv4-address-object", > "value":"1.2.3.4" > }, > "1":{ > "type":"ipv4-address-object", > "value":"2.3.4.5" > }, > "2":{ > "type":"network-connection-object", > "src_ref":"0", > "dst_ref":"1", > "protocols":[ "ipv4", "tcp-closed" ] > } > } > } > > > From: Bret Jordan <bret.jordan@bluecoat.com> > > Date: Tuesday, August 30, 2016 at 4:56 PM > > To: "Wunder, John A." <jwunder@mitre.org> > > Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> > > Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] > > > > For network-connection though, there are time fields in CybOX land. So how do those relate to the time stamps in the Observed Data object.??? > > > > Thanks, > > > > Bret > > > > > > > > Bret Jordan CISSP > > Director of Security Architecture and Standards Office of the CTO > > Blue Coat Systems > > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > > > On Aug 30, 2016, at 14:28, Wunder, John A. <jwunder@mitre.org< mailto:jwunder@mitre.org >> wrote: > > > > When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor. > > > > On the issues themselves: > > - The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. > > - I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. > > John > > > > From: <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> on behalf of Bret Jordan <bret.jordan@bluecoat.com< mailto:bret.jordan@bluecoat.com >> > > Date: Tuesday, August 30, 2016 at 4:10 PM > > To: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > > Cc: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > > Subject: Re: [cti-stix] [jmg@newcontext.com< mailto:jmg@newcontext.com >: Observed Data comments.] > > > > How should we address this? Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot. Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review. > > > > Thanks, > > > > Bret > > > > > > > > Bret Jordan CISSP > > Director of Security Architecture and Standards Office of the CTO > > Blue Coat Systems > > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > > > On Aug 30, 2016, at 12:25, John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> wrote: > > > > Did this get lost in the shuffle last week? I didn't see any discussion > > on this. > > > > ----- Forwarded message from John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> ----- > > > > Date: Mon, 22 Aug 2016 17:08:44 -0700 > > From: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > > To: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > > Subject: Observed Data comments. > > > > Hello, > > > > Sorry for not getting this in sooner, but I have questions/comments > > about the Observed Data object. > > > > There is nothing in the spec that requires first_observed to be equal > > to last_observed when number_observed is 1. > > > > How is a tool who receives such an object to handle this? > > > > In the case of when you don't know exactly when the observed data > > was recorded, the _presision property added to first_observed can > > convey this information correctly, and most acurately. > > > > P.S. IMO, we should also make the number_observed default to 1, and > > make it optional. By setting a resonable default, we can reduce > > the size of objects in common cases. > > > > -- > > John-Mark > > > > ----- End forwarded message ----- > > -- > John-Mark > > > -- John-Mark --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php

"Having each network connection require two Observed Data instances to represent the start and the stop seems kind of crazy to me. I’m not sure tools would get that right and having double the messages is going to be an issue" While I agree that if you wanted to use CybOX to send alerts on an active network connection from a tool before you decide to terminate it that you would run into this scenario, but I'm confused why this is a major use case. It seems like an IPS would use its own native reporting format to determine if a connection should be blocked rather than STIX, which is inherently less detailed. Right now the bulk of the network connection objects I have seen in CybOX come from sandboxing tools that have the luxury of waiting for a connection to run its course or from pcap data that does the same. In both cases we only need to produce a single Observed Data instance. Jeffrey Mates, Civ DC3/DCCI ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Computer Scientist Defense Cyber Crime Institute jeffrey.mates@dc3.mil 410-694-4335
Original Message----- From: cti-stix@lists.oasis-open.org [ mailto:cti-stix@lists.oasis-open.org ] On Behalf Of Wunder, John A. Sent: Tuesday, August 30, 2016 7:27 PM To: cti-stix@lists.oasis-open.org Subject: [Non-DoD Source] Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] So here’s my perspective. I’m not super involved in patterning so take those portions with a grain of salt, and to be honest I don’t feel super strongly about any of this. I just want to explain why I think what we have works semantically. - Having each network connection require two Observed Data instances to represent the start and the stop seems kind of crazy to me. I’m not sure tools would get that right and having double the messages is going to be an issue. - Matching is rarely going to occur against literal Observed Data instances in actual products (IMO). STIX Observed Data is a serialization format. Matching can be written as if it’s against Observed Data if that helps explain how it works, but the raw data is the raw data. Making Observed Data have different semantics to make matching easier seems to be the wrong way to go about this to me. Patterning needs an approach to matching network connections (and other events) that occur over time ranges and figuring out some simple approach for that would absolutely make sense, but that doesn’t need to change the semantics of observed data. Maybe you can have some normative text in the patterning specification that describes how temporal verbs match against time windows. - I don’t think what I said was multiple ways to do the same thing. Observed Data captures the metadata about an observation, including the time that the observation was captured. If I observe a network connection that just opened then my first/last observed will be the same because I observed that network connection over that time. If I observe it a minute later when it’s still open, then my first observed is when I first started capturing the data and my last observed is when I last started capturing it. If I observe it when it’s closed, the first observed is when you first started capturing the data and the last observed is when you finished capturing that data, aka when the connection closed. These are all just observations of the same network connection, just as you can have multiple observations (observed data) of the same file. If you want to write a pattern to match that network connection you need to assume that the connection occurs over a time window, because it does. Some sensors might output multiple observations of that connection, others might just report one when it’s over, but the raw data you’re matching against is the same either way. John On 8/30/16, 6:05 PM, "cti-stix@lists.oasis-open.org on behalf of John-Mark Gurney" <cti-stix@lists.oasis-open.org on behalf of jmg@newcontext.com> wrote: Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:46 +0000: > Sorry I had a paragraph here that I removed in my rush to get home: sending a network connection opening, as a discrete event, would make sense to me and is supported now (make the timestamps the same). I just don't think we should make discrete start/stop or open/close events the ONLY way to do it by requiring they be the same for single counts. If we end up w/ multiple ways to do the same thing, then writing a pattern will not work genericly. We'll end up w/ a pattern that only works when it used w/ data from sensor A, but not B because of the different formats for the same information. John-Mark > From: Wunder, John A. <jwunder@mitre.org< mailto:jwunder@mitre.org >> > Date: Tuesday, Aug 30, 2016, 5:38 PM > To: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > Cc: Jordan, Bret <bret.jordan@bluecoat.com< mailto:bret.jordan@bluecoat.com >>, cti-stix@lists.oasis-open.org <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] > > I guess my point of view is that some observations just by nature inherently occur over a period of time and we need to deal with it. Yes, you can send a connection opened event and a connection closed event but in reality for matching and analysis you’re going to need to operate over that entire time window. Same thing even for file create (which can take time) or more complex actions. Some things occur over a time window and IMO we need to capture that time window in Observed Data. > > John > > > On 8/30/16, 5:27 PM, "John-Mark Gurney" <jmg@newcontext.com> wrote: > > Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:12 +0000: > > The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection. > > > > How do you think it should work? > > If a sensor delays sending out the object until connection closes, it > means that this data is not available to decide if action should be > taken. This leaves a huge gap in coverage. > > IMO, if Observed Data isn't a single point in time, it will become more > complicated to analyze the data. > > Representing connection open/closed can be easily recorded w/ two > objects recording the state, the first one w/ state of CONNECTED, > and a second one w/ the state CLOSED. As the four-tuple is required > to be unique, this won't cause any confusion between connections unless > objects are lost. > > I used -connected and -closed because CybOX says: > Specifies the protocols used in the network connection, along with their corresponding state. > > But provides no examples of how to convey the state information, and > the TCP Extension does not have it, though it could possibly done > w/ flags (via SYN/FIN). > > { > "type":"cybox-container", > "spec_version":"3.0", > "objects":{ > "0":{ > "type":"ipv4-address-object", > "value":"1.2.3.4" > }, > "1":{ > "type":"ipv4-address-object", > "value":"2.3.4.5" > }, > "2":{ > "type":"network-connection-object", > "src_ref":"0", > "dst_ref":"1", > "protocols":[ "ipv4", "tcp-connected" ] > } > } > } > > { > "type":"cybox-container", > "spec_version":"3.0", > "objects":{ > "0":{ > "type":"ipv4-address-object", > "value":"1.2.3.4" > }, > "1":{ > "type":"ipv4-address-object", > "value":"2.3.4.5" > }, > "2":{ > "type":"network-connection-object", > "src_ref":"0", > "dst_ref":"1", > "protocols":[ "ipv4", "tcp-closed" ] > } > } > } > > > From: Bret Jordan <bret.jordan@bluecoat.com> > > Date: Tuesday, August 30, 2016 at 4:56 PM > > To: "Wunder, John A." <jwunder@mitre.org> > > Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> > > Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] > > > > For network-connection though, there are time fields in CybOX land. So how do those relate to the time stamps in the Observed Data object.??? > > > > Thanks, > > > > Bret > > > > > > > > Bret Jordan CISSP > > Director of Security Architecture and Standards Office of the CTO > > Blue Coat Systems > > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > > > On Aug 30, 2016, at 14:28, Wunder, John A. <jwunder@mitre.org< mailto:jwunder@mitre.org >> wrote: > > > > When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor. > > > > On the issues themselves: > > - The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. > > - I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. > > John > > > > From: <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> on behalf of Bret Jordan <bret.jordan@bluecoat.com< mailto:bret.jordan@bluecoat.com >> > > Date: Tuesday, August 30, 2016 at 4:10 PM > > To: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > > Cc: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > > Subject: Re: [cti-stix] [jmg@newcontext.com< mailto:jmg@newcontext.com >: Observed Data comments.] > > > > How should we address this? Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot. Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review. > > > > Thanks, > > > > Bret > > > > > > > > Bret Jordan CISSP > > Director of Security Architecture and Standards Office of the CTO > > Blue Coat Systems > > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > > > On Aug 30, 2016, at 12:25, John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> wrote: > > > > Did this get lost in the shuffle last week? I didn't see any discussion > > on this. > > > > ----- Forwarded message from John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> ----- > > > > Date: Mon, 22 Aug 2016 17:08:44 -0700 > > From: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > > To: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > > Subject: Observed Data comments. > > > > Hello, > > > > Sorry for not getting this in sooner, but I have questions/comments > > about the Observed Data object. > > > > There is nothing in the spec that requires first_observed to be equal > > to last_observed when number_observed is 1. > > > > How is a tool who receives such an object to handle this? > > > > In the case of when you don't know exactly when the observed data > > was recorded, the _presision property added to first_observed can > > convey this information correctly, and most acurately. > > > > P.S. IMO, we should also make the number_observed default to 1, and > > make it optional. By setting a resonable default, we can reduce > > the size of objects in common cases. > > > > -- > > John-Mark > > > > ----- End forwarded message ----- > > -- > John-Mark > > > -- John-Mark --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php Attachment: smime.p7s Description: S/MIME cryptographic signature

[[[So although I will continue to argue for certain views, *Please* accept this comment in the humorous light intended ("If you cant laugh at yourself, who can you laugh at?"). ]]] "Having each network connection require two Observed Data instances to represent the start and the stop seems kind of crazy to me" I can't pass up a chance to talk about TimeStamps.

I think one thing that John-Mark you are getting confused on is Patterning works against CybOX data not STIX data. So you would never be able to write a batter against an Observed Data object as that is STIX. Bret Sent from my Commodore 64 > On Aug 30, 2016, at 5:26 PM, Wunder, John A. <jwunder@mitre.org> wrote: > > So here’s my perspective. I’m not super involved in patterning so take those portions with a grain of salt, and to be honest I don’t feel super strongly about any of this. I just want to explain why I think what we have works semantically. > > - Having each network connection require two Observed Data instances to represent the start and the stop seems kind of crazy to me. I’m not sure tools would get that right and having double the messages is going to be an issue. > - Matching is rarely going to occur against literal Observed Data instances in actual products (IMO). STIX Observed Data is a serialization format. Matching can be written as if it’s against Observed Data if that helps explain how it works, but the raw data is the raw data. Making Observed Data have different semantics to make matching easier seems to be the wrong way to go about this to me. Patterning needs an approach to matching network connections (and other events) that occur over time ranges and figuring out some simple approach for that would absolutely make sense, but that doesn’t need to change the semantics of observed data. Maybe you can have some normative text in the patterning specification that describes how temporal verbs match against time windows. > - I don’t think what I said was multiple ways to do the same thing. Observed Data captures the metadata about an observation, including the time that the observation was captured. If I observe a network connection that just opened then my first/last observed will be the same because I observed that network connection over that time. If I observe it a minute later when it’s still open, then my first observed is when I first started capturing the data and my last observed is when I last started capturing it. If I observe it when it’s closed, the first observed is when you first started capturing the data and the last observed is when you finished capturing that data, aka when the connection closed. These are all just observations of the same network connection, just as you can have multiple observations (observed data) of the same file. If you want to write a pattern to match that network connection you need to assume that the connection occurs over a time window, because it does. Some sensors might output multiple observations of that connection, others might just report one when it’s over, but the raw data you’re matching against is the same either way. > > John > > On 8/30/16, 6:05 PM, "cti-stix@lists.oasis-open.org on behalf of John-Mark Gurney" <cti-stix@lists.oasis-open.org on behalf of jmg@newcontext.com> wrote: > > Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:46 +0000: >> Sorry I had a paragraph here that I removed in my rush to get home: sending a network connection opening, as a discrete event, would make sense to me and is supported now (make the timestamps the same). I just don't think we should make discrete start/stop or open/close events the ONLY way to do it by requiring they be the same for single counts. > > If we end up w/ multiple ways to do the same thing, then writing a > pattern will not work genericly. We'll end up w/ a pattern that only > works when it used w/ data from sensor A, but not B because of the > different formats for the same information. > > John-Mark > >> From: Wunder, John A. <jwunder@mitre.org< mailto:jwunder@mitre.org >> >> Date: Tuesday, Aug 30, 2016, 5:38 PM >> To: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> >> Cc: Jordan, Bret <bret.jordan@bluecoat.com< mailto:bret.jordan@bluecoat.com >>, cti-stix@lists.oasis-open.org <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> >> Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] >> >> I guess my point of view is that some observations just by nature inherently occur over a period of time and we need to deal with it. Yes, you can send a connection opened event and a connection closed event but in reality for matching and analysis you’re going to need to operate over that entire time window. Same thing even for file create (which can take time) or more complex actions. Some things occur over a time window and IMO we need to capture that time window in Observed Data. >> >> John >> >> >> On 8/30/16, 5:27 PM, "John-Mark Gurney" <jmg@newcontext.com> wrote: >> >> Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:12 +0000: >>> The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection. >>> >>> How do you think it should work? >> >> If a sensor delays sending out the object until connection closes, it >> means that this data is not available to decide if action should be >> taken. This leaves a huge gap in coverage. >> >> IMO, if Observed Data isn't a single point in time, it will become more >> complicated to analyze the data. >> >> Representing connection open/closed can be easily recorded w/ two >> objects recording the state, the first one w/ state of CONNECTED, >> and a second one w/ the state CLOSED. As the four-tuple is required >> to be unique, this won't cause any confusion between connections unless >> objects are lost. >> >> I used -connected and -closed because CybOX says: >> Specifies the protocols used in the network connection, along with their corresponding state. >> >> But provides no examples of how to convey the state information, and >> the TCP Extension does not have it, though it could possibly done >> w/ flags (via SYN/FIN). >> >> { >> "type":"cybox-container", >> "spec_version":"3.0", >> "objects":{ >> "0":{ >> "type":"ipv4-address-object", >> "value":"1.2.3.4" >> }, >> "1":{ >> "type":"ipv4-address-object", >> "value":"2.3.4.5" >> }, >> "2":{ >> "type":"network-connection-object", >> "src_ref":"0", >> "dst_ref":"1", >> "protocols":[ "ipv4", "tcp-connected" ] >> } >> } >> } >> >> { >> "type":"cybox-container", >> "spec_version":"3.0", >> "objects":{ >> "0":{ >> "type":"ipv4-address-object", >> "value":"1.2.3.4" >> }, >> "1":{ >> "type":"ipv4-address-object", >> "value":"2.3.4.5" >> }, >> "2":{ >> "type":"network-connection-object", >> "src_ref":"0", >> "dst_ref":"1", >> "protocols":[ "ipv4", "tcp-closed" ] >> } >> } >> } >> >>> From: Bret Jordan <bret.jordan@bluecoat.com> >>> Date: Tuesday, August 30, 2016 at 4:56 PM >>> To: "Wunder, John A." <jwunder@mitre.org> >>> Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> >>> Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] >>> >>> For network-connection though, there are time fields in CybOX land. So how do those relate to the time stamps in the Observed Data object.??? >>> >>> Thanks, >>> >>> Bret >>> >>> >>> >>> Bret Jordan CISSP >>> Director of Security Architecture and Standards Office of the CTO >>> Blue Coat Systems >>> PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 >>> "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." >>> >>> On Aug 30, 2016, at 14:28, Wunder, John A. <jwunder@mitre.org< mailto:jwunder@mitre.org >> wrote: >>> >>> When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor. >>> >>> On the issues themselves: >>> - The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. >>> - I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. >>> John >>> >>> From: <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> on behalf of Bret Jordan <bret.jordan@bluecoat.com< mailto:bret.jordan@bluecoat.com >> >>> Date: Tuesday, August 30, 2016 at 4:10 PM >>> To: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> >>> Cc: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> >>> Subject: Re: [cti-stix] [jmg@newcontext.com< mailto:jmg@newcontext.com >: Observed Data comments.] >>> >>> How should we address this? Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot. Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review. >>> >>> Thanks, >>> >>> Bret >>> >>> >>> >>> Bret Jordan CISSP >>> Director of Security Architecture and Standards Office of the CTO >>> Blue Coat Systems >>> PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 >>> "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." >>> >>> On Aug 30, 2016, at 12:25, John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> wrote: >>> >>> Did this get lost in the shuffle last week? I didn't see any discussion >>> on this. >>> >>> ----- Forwarded message from John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> ----- >>> >>> Date: Mon, 22 Aug 2016 17:08:44 -0700 >>> From: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> >>> To: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> >>> Subject: Observed Data comments. >>> >>> Hello, >>> >>> Sorry for not getting this in sooner, but I have questions/comments >>> about the Observed Data object. >>> >>> There is nothing in the spec that requires first_observed to be equal >>> to last_observed when number_observed is 1. >>> >>> How is a tool who receives such an object to handle this? >>> >>> In the case of when you don't know exactly when the observed data >>> was recorded, the _presision property added to first_observed can >>> convey this information correctly, and most acurately. >>> >>> P.S. IMO, we should also make the number_observed default to 1, and >>> make it optional. By setting a resonable default, we can reduce >>> the size of objects in common cases. >>> >>> -- >>> John-Mark >>> >>> ----- End forwarded message ----- >> >> -- >> John-Mark >> >> >> > > -- > John-Mark > > --------------------------------------------------------------------- > To unsubscribe from this mail list, you must leave the OASIS TC that > generates this mail. Follow this link to all your TCs in OASIS at: > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php > > > >

Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:37 +0000: > I guess my point of view is that some observations just by nature inherently occur over a period of time and we need to deal with it. Yes, you can send a connection opened event and a connection closed event but in reality for matching and analysis you’re going to need to operate over that entire time window. Same thing even for file create (which can take time) or more complex actions. Some things occur over a time window and IMO we need to capture that time window in Observed Data. That what you'd do w/ the future back reference capabilities: a = [ network-connection-object:protocols[*] = 'tcp-connected' ] FOLLOWEDBY [ network-connection-object:src = a:src AND network-connection-object:dst = a:dst AND network-connection-object:src_port = a:src_port AND network-connection-object:dst_port = a:dst_port AND network-connection-boject:protocols[*] = 'tcp-closed' ] So, if we now have to support two time stamps for Observed Data, then we need to replace the FOLLOWEDBY operator w/ three new ones: AFTEROVERLAPS (b.start > a.start AND b.start < a.end), AFTERDISJOINT (b.start > a.start AND b.start > a.end) and AFTERCONTAINS (b.start > a.start, b.end < a.end) To handle all the possible cases that these two timestamps adds, or we can have patterning stick it's head in the group wrt to the second time stamp, and say that patterning only ever deals w/ the start time stamp. John-Mark > On 8/30/16, 5:27 PM, "John-Mark Gurney" <jmg@newcontext.com> wrote: > > Wunder, John A. wrote this message on Tue, Aug 30, 2016 at 21:12 +0000: > > The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection. > > > > How do you think it should work? > > If a sensor delays sending out the object until connection closes, it > means that this data is not available to decide if action should be > taken. This leaves a huge gap in coverage. > > IMO, if Observed Data isn't a single point in time, it will become more > complicated to analyze the data. > > Representing connection open/closed can be easily recorded w/ two > objects recording the state, the first one w/ state of CONNECTED, > and a second one w/ the state CLOSED. As the four-tuple is required > to be unique, this won't cause any confusion between connections unless > objects are lost. > > I used -connected and -closed because CybOX says: > Specifies the protocols used in the network connection, along with their corresponding state. > > But provides no examples of how to convey the state information, and > the TCP Extension does not have it, though it could possibly done > w/ flags (via SYN/FIN). > > { > "type":"cybox-container", > "spec_version":"3.0", > "objects":{ > "0":{ > "type":"ipv4-address-object", > "value":"1.2.3.4" > }, > "1":{ > "type":"ipv4-address-object", > "value":"2.3.4.5" > }, > "2":{ > "type":"network-connection-object", > "src_ref":"0", > "dst_ref":"1", > "protocols":[ "ipv4", "tcp-connected" ] > } > } > } > > { > "type":"cybox-container", > "spec_version":"3.0", > "objects":{ > "0":{ > "type":"ipv4-address-object", > "value":"1.2.3.4" > }, > "1":{ > "type":"ipv4-address-object", > "value":"2.3.4.5" > }, > "2":{ > "type":"network-connection-object", > "src_ref":"0", > "dst_ref":"1", > "protocols":[ "ipv4", "tcp-closed" ] > } > } > } > > > From: Bret Jordan <bret.jordan@bluecoat.com> > > Date: Tuesday, August 30, 2016 at 4:56 PM > > To: "Wunder, John A." <jwunder@mitre.org> > > Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> > > Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] > > > > For network-connection though, there are time fields in CybOX land. So how do those relate to the time stamps in the Observed Data object.??? > > > > Thanks, > > > > Bret > > > > > > > > Bret Jordan CISSP > > Director of Security Architecture and Standards Office of the CTO > > Blue Coat Systems > > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > > > On Aug 30, 2016, at 14:28, Wunder, John A. <jwunder@mitre.org< mailto:jwunder@mitre.org >> wrote: > > > > When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor. > > > > On the issues themselves: > > - The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. > > - I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. > > John > > > > From: <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> on behalf of Bret Jordan <bret.jordan@bluecoat.com< mailto:bret.jordan@bluecoat.com >> > > Date: Tuesday, August 30, 2016 at 4:10 PM > > To: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > > Cc: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > > Subject: Re: [cti-stix] [jmg@newcontext.com< mailto:jmg@newcontext.com >: Observed Data comments.] > > > > How should we address this? Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot. Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review. > > > > Thanks, > > > > Bret > > > > > > > > Bret Jordan CISSP > > Director of Security Architecture and Standards Office of the CTO > > Blue Coat Systems > > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 > > "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." > > > > On Aug 30, 2016, at 12:25, John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> wrote: > > > > Did this get lost in the shuffle last week? I didn't see any discussion > > on this. > > > > ----- Forwarded message from John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> ----- > > > > Date: Mon, 22 Aug 2016 17:08:44 -0700 > > From: John-Mark Gurney <jmg@newcontext.com< mailto:jmg@newcontext.com >> > > To: "cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >" <cti-stix@lists.oasis-open.org< mailto:cti-stix@lists.oasis-open.org >> > > Subject: Observed Data comments. > > > > Hello, > > > > Sorry for not getting this in sooner, but I have questions/comments > > about the Observed Data object. > > > > There is nothing in the spec that requires first_observed to be equal > > to last_observed when number_observed is 1. > > > > How is a tool who receives such an object to handle this? > > > > In the case of when you don't know exactly when the observed data > > was recorded, the _presision property added to first_observed can > > convey this information correctly, and most acurately. > > > > P.S. IMO, we should also make the number_observed default to 1, and > > make it optional. By setting a resonable default, we can reduce > > the size of objects in common cases. > > > > -- > > John-Mark > > > > ----- End forwarded message ----- -- John-Mark

Actually... In the case of a network flow they would be expected to be different. Flows have distinct start and end times. -- Sent from my mobile device, please excuse any typos. Wunder, John A. --- Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] --- From: "Wunder, John A." <jwunder@mitre.org> To: "Jordan, Bret" <bret.jordan@bluecoat.com> Cc: "John-Mark Gurney" <jmg@newcontext.com>, cti-stix@lists.oasis-open.org Date: Tue, Aug 30, 2016 6:12 PM Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection.   How do you think it should work?   From: Bret Jordan <bret.jordan@bluecoat.com> Date: Tuesday, August 30, 2016 at 4:56 PM To: "Wunder, John A." <jwunder@mitre.org> Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]   For network-connection though, there are time fields in CybOX land.  So how do those relate to the time stamps in the Observed Data object.???    Thanks,   Bret       Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."    On Aug 30, 2016, at 14:28, Wunder, John A. < jwunder@mitre.org > wrote:   When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor.   On the issues themselves: -             The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. -             I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. John   From:   < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date:   Tuesday, August 30, 2016 at 4:10 PM To:   John-Mark Gurney < jmg@newcontext.com > Cc:   " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject:   Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   How should we address this?  Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot.  Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review.   Thanks,   Bret       Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."    On Aug 30, 2016, at 12:25, John-Mark Gurney < jmg@newcontext.com > wrote:   Did this get lost in the shuffle last week?  I didn't see any discussion on this. ----- Forwarded message from John-Mark Gurney < jmg@newcontext.com > ----- Date: Mon, 22 Aug 2016 17:08:44 -0700 From: John-Mark Gurney < jmg@newcontext.com > To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Observed Data comments. Hello, Sorry for not getting this in sooner, but I have questions/comments about the Observed Data object. There is nothing in the spec that requires first_observed to be equal to last_observed when number_observed is 1. How is a tool who receives such an object to handle this? In the case of when you don't know exactly when the observed data was recorded, the _presision property added to first_observed can convey this information correctly, and most acurately. P.S. IMO, we should also make the number_observed default to 1, and make it optional.  By setting a resonable default, we can reduce the size of objects in common cases. --   John-Mark ----- End forwarded message ----- --   John-Mark --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that   generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php    

I don’t really follow what you’re saying here …we have a bunch of timestamps to consider:
 
-          
Network Connection/Flow Start (time the connection opened)
-          
Network Connection/Flow End (time the connection closed)
-          
Observed Data Start (time the observed data was first observed)
-          
Observed Data End (time the observed data was last observed)
 
My point was just that Observed Data Start time (time you started collecting the observed data) should match the network connection start time (time the connection opened) and the Observed
Data End Time (time you finished collecting the observed data) should match the network connection end time (time the connection closed).

 
I was suggesting that we should not require the Observed Data Start/End times to be the same when you only have one observation because sometimes single observations (network connections
being the best example) occur over some time period.
 
John
 

From:
Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Date: Thursday, September 1, 2016 at 4:53 PM
To: "Wunder, John A." <jwunder@mitre.org>
Cc: Bret Jordan <bret.jordan@bluecoat.com>, John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]


 



Actually... In the case of a network flow they would be expected to be different. Flows have distinct start and end times.

--
Sent from my mobile device, please excuse any typos.


Wunder, John A. --- Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] ---



 




From:


"Wunder, John A." <jwunder@mitre.org>




To:


"Jordan, Bret" <bret.jordan@bluecoat.com>




Cc:


"John-Mark Gurney" <jmg@newcontext.com>, cti-stix@lists.oasis-open.org




Date:


Tue, Aug 30, 2016 6:12 PM




Subject:


Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]







 

The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed
and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection.
 
How do you think it should work?
 

From:
Bret Jordan <bret.jordan@bluecoat.com>
Date: Tuesday, August 30, 2016 at 4:56 PM
To: "Wunder, John A." <jwunder@mitre.org>
Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]


 



For network-connection though, there are time fields in CybOX land.  So how do those relate to the time stamps in the Observed Data object.??? 







 


Thanks,


 


Bret



 


 


 



Bret Jordan CISSP

Director of Security Architecture and Standards Office of the CTO


Blue Coat Systems



PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050


"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 









 



On Aug 30, 2016, at 14:28, Wunder, John A. < jwunder@mitre.org > wrote:

 


When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway
since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor.


 


On the issues themselves:


-             The
reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have
a requirement that they be the same when the count is one.


-             I
would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this.


John


 



From:   < cti-stix@lists.oasis-open.org >
on behalf of Bret Jordan < bret.jordan@bluecoat.com >
Date:   Tuesday, August 30, 2016 at 4:10 PM
To:   John-Mark Gurney < jmg@newcontext.com >
Cc:   " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
Subject:   Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]




 





How should we address this?  Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot.  Keep in mind we can do as many Committee Specification Draft releases
we want before we go to a Committee Specification with Public Review.








 



Thanks,




 




Bret





 




 




 





Bret Jordan CISSP



Director of Security Architecture and Standards Office of the CTO




Blue Coat Systems





PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050




"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 











 





On Aug 30, 2016, at 12:25, John-Mark Gurney < jmg@newcontext.com > wrote:



 



Did this get lost in the shuffle last week?  I didn't see any discussion
on this.

----- Forwarded message from John-Mark Gurney < jmg@newcontext.com > -----

Date: Mon, 22 Aug 2016 17:08:44 -0700
From: John-Mark Gurney < jmg@newcontext.com >
To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
Subject: Observed Data comments.

Hello,

Sorry for not getting this in sooner, but I have questions/comments
about the Observed Data object.

There is nothing in the spec that requires first_observed to be equal
to last_observed when number_observed is 1.

How is a tool who receives such an object to handle this?

In the case of when you don't know exactly when the observed data
was recorded, the _presision property added to first_observed can
convey this information correctly, and most acurately.

P.S. IMO, we should also make the number_observed default to 1, and
make it optional.  By setting a resonable default, we can reduce
the size of objects in common cases.

--  
John-Mark

----- End forwarded message -----

--  
John-Mark

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that  
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php





 






 


 

John – I agree with you if that is what you want to convey.
 
That is, I observed something for X time starting Y and ending Z and that coincides with 1 flow with the exact same timestamps.
 
However, if I want to convey that I watched for a pattern for 24 hours and I only saw 1 flow then the observed data timestamps would be when I watched for any activity and the network connection
timestamps would be for the 1 flow that occurred.
 
So in that case they would be necessarily different.
 
allan
 

From:
"cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> on behalf of "Wunder, John" <jwunder@mitre.org>
Date: Thursday, September 1, 2016 at 2:20 PM
To: Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Cc: "Jordan, Bret" <bret.jordan@bluecoat.com>, John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]


 



I don’t really follow what you’re saying here …we have a bunch of timestamps to consider:
 
-          
Network Connection/Flow Start (time the connection opened)
-          
Network Connection/Flow End (time the connection closed)
-          
Observed Data Start (time the observed data was first observed)
-          
Observed Data End (time the observed data was last observed)
 
My point was just that Observed Data Start time (time you started collecting the observed data) should match the network connection start time (time the connection opened) and the Observed
Data End Time (time you finished collecting the observed data) should match the network connection end time (time the connection closed).

 
I was suggesting that we should not require the Observed Data Start/End times to be the same when you only have one observation because sometimes single observations (network connections
being the best example) occur over some time period.
 
John
 

From:
Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Date: Thursday, September 1, 2016 at 4:53 PM
To: "Wunder, John A." <jwunder@mitre.org>
Cc: Bret Jordan <bret.jordan@bluecoat.com>, John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]


 



Actually... In the case of a network flow they would be expected to be different. Flows have distinct start and end times.

--
Sent from my mobile device, please excuse any typos.


Wunder, John A. --- Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] ---



 




From:


"Wunder, John A." <jwunder@mitre.org>




To:


"Jordan, Bret" <bret.jordan@bluecoat.com>




Cc:


"John-Mark Gurney" <jmg@newcontext.com>, cti-stix@lists.oasis-open.org




Date:


Tue, Aug 30, 2016 6:12 PM




Subject:


Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]









 

The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed
and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection.
 
How do you think it should work?
 

From:
Bret Jordan <bret.jordan@bluecoat.com>
Date: Tuesday, August 30, 2016 at 4:56 PM
To: "Wunder, John A." <jwunder@mitre.org>
Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]


 



For network-connection though, there are time fields in CybOX land.  So how do those relate to the time stamps in the Observed Data object.??? 







 


Thanks,


 


Bret



 


 


 



Bret Jordan CISSP

Director of Security Architecture and Standards Office of the CTO


Blue Coat Systems



PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050


"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 









 



On Aug 30, 2016, at 14:28, Wunder, John A. < jwunder@mitre.org > wrote:

 


When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway
since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor.


 


On the issues themselves:


-             The
reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have
a requirement that they be the same when the count is one.


-             I
would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this.


John


 



From:   < cti-stix@lists.oasis-open.org >
on behalf of Bret Jordan < bret.jordan@bluecoat.com >
Date:   Tuesday, August 30, 2016 at 4:10 PM
To:   John-Mark Gurney < jmg@newcontext.com >
Cc:   " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
Subject:   Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]




 





How should we address this?  Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot.  Keep in mind we can do as many Committee Specification Draft releases
we want before we go to a Committee Specification with Public Review.








 



Thanks,




 




Bret





 




 




 





Bret Jordan CISSP



Director of Security Architecture and Standards Office of the CTO




Blue Coat Systems





PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050




"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 











 





On Aug 30, 2016, at 12:25, John-Mark Gurney < jmg@newcontext.com > wrote:



 



Did this get lost in the shuffle last week?  I didn't see any discussion
on this.

----- Forwarded message from John-Mark Gurney < jmg@newcontext.com > -----

Date: Mon, 22 Aug 2016 17:08:44 -0700
From: John-Mark Gurney < jmg@newcontext.com >
To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
Subject: Observed Data comments.

Hello,

Sorry for not getting this in sooner, but I have questions/comments
about the Observed Data object.

There is nothing in the spec that requires first_observed to be equal
to last_observed when number_observed is 1.

How is a tool who receives such an object to handle this?

In the case of when you don't know exactly when the observed data
was recorded, the _presision property added to first_observed can
convey this information correctly, and most acurately.

P.S. IMO, we should also make the number_observed default to 1, and
make it optional.  By setting a resonable default, we can reduce
the size of objects in common cases.

--  
John-Mark

----- End forwarded message -----

--  
John-Mark

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that  
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php





 






 


 

Allan,  that is how I thought about it too. Bret  Sent from my Commodore 64 On Sep 1, 2016, at 3:31 PM, Allan Thomson < athomson@lookingglasscyber.com > wrote: John – I agree with you if that is what you want to convey.   That is, I observed something for X time starting Y and ending Z and that coincides with 1 flow with the exact same timestamps.   However, if I want to convey that I watched for a pattern for 24 hours and I only saw 1 flow then the observed data timestamps would be when I watched for any activity and the network connection timestamps would be for the 1 flow that occurred.   So in that case they would be necessarily different.   allan   From: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > on behalf of "Wunder, John" < jwunder@mitre.org > Date: Thursday, September 1, 2016 at 2:20 PM To: Jason Keirstead < Jason.Keirstead@ca.ibm.com > Cc: "Jordan, Bret" < bret.jordan@bluecoat.com >, John-Mark Gurney < jmg@newcontext.com >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   I don’t really follow what you’re saying here …we have a bunch of timestamps to consider:   -           Network Connection/Flow Start (time the connection opened) -           Network Connection/Flow End (time the connection closed) -           Observed Data Start (time the observed data was first observed) -           Observed Data End (time the observed data was last observed)   My point was just that Observed Data Start time (time you started collecting the observed data) should match the network connection start time (time the connection opened) and the Observed Data End Time (time you finished collecting the observed data) should match the network connection end time (time the connection closed).   I was suggesting that we should not require the Observed Data Start/End times to be the same when you only have one observation because sometimes single observations (network connections being the best example) occur over some time period.   John   From: Jason Keirstead < Jason.Keirstead@ca.ibm.com > Date: Thursday, September 1, 2016 at 4:53 PM To: "Wunder, John A." < jwunder@mitre.org > Cc: Bret Jordan < bret.jordan@bluecoat.com >, John-Mark Gurney < jmg@newcontext.com >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   Actually... In the case of a network flow they would be expected to be different. Flows have distinct start and end times. -- Sent from my mobile device, please excuse any typos. Wunder, John A. --- Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.] ---   From: "Wunder, John A." < jwunder@mitre.org > To: "Jordan, Bret" < bret.jordan@bluecoat.com > Cc: "John-Mark Gurney" < jmg@newcontext.com >, cti-stix@lists.oasis-open.org Date: Tue, Aug 30, 2016 6:12 PM Subject: Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection.   How do you think it should work?   From: Bret Jordan < bret.jordan@bluecoat.com > Date: Tuesday, August 30, 2016 at 4:56 PM To: "Wunder, John A." < jwunder@mitre.org > Cc: John-Mark Gurney < jmg@newcontext.com >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   For network-connection though, there are time fields in CybOX land.  So how do those relate to the time stamps in the Observed Data object.???    Thanks,   Bret       Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."    On Aug 30, 2016, at 14:28, Wunder, John A. < jwunder@mitre.org > wrote:   When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor.   On the issues themselves: -             The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. -             I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. John   From:   < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date:   Tuesday, August 30, 2016 at 4:10 PM To:   John-Mark Gurney < jmg@newcontext.com > Cc:   " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject:   Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   How should we address this?  Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot.  Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review.   Thanks,   Bret       Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."    On Aug 30, 2016, at 12:25, John-Mark Gurney < jmg@newcontext.com > wrote:   Did this get lost in the shuffle last week?  I didn't see any discussion on this. ----- Forwarded message from John-Mark Gurney < jmg@newcontext.com > ----- Date: Mon, 22 Aug 2016 17:08:44 -0700 From: John-Mark Gurney < jmg@newcontext.com > To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Observed Data comments. Hello, Sorry for not getting this in sooner, but I have questions/comments about the Observed Data object. There is nothing in the spec that requires first_observed to be equal to last_observed when number_observed is 1. How is a tool who receives such an object to handle this? In the case of when you don't know exactly when the observed data was recorded, the _presision property added to first_observed can convey this information correctly, and most acurately. P.S. IMO, we should also make the number_observed default to 1, and make it optional.  By setting a resonable default, we can reduce the size of objects in common cases. --   John-Mark ----- End forwarded message ----- --   John-Mark --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that   generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php      

OK, clarified,  I agree, They should not have to be the same. Discrete observations can have start and end times. Off the top of my head, in addition to Network flows, Process executions (malware analysis) also have a start and end time -- Sent from my mobile device, please excuse any typos. Wunder, John A. --- Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] --- From: "Wunder, John A." <jwunder@mitre.org> To: "Jason Keirstead" <Jason.Keirstead@ca.ibm.com> Cc: "Jordan, Bret" <bret.jordan@bluecoat.com>, "John-Mark Gurney" <jmg@newcontext.com>, cti-stix@lists.oasis-open.org Date: Thu, Sep 1, 2016 6:20 PM Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] I don’t really follow what you’re saying here …we have a bunch of timestamps to consider:   -           Network Connection/Flow Start (time the connection opened) -           Network Connection/Flow End (time the connection closed) -           Observed Data Start (time the observed data was first observed) -           Observed Data End (time the observed data was last observed)   My point was just that Observed Data Start time (time you started collecting the observed data) should match the network connection start time (time the connection opened) and the Observed Data End Time (time you finished collecting the observed data) should match the network connection end time (time the connection closed).   I was suggesting that we should not require the Observed Data Start/End times to be the same when you only have one observation because sometimes single observations (network connections being the best example) occur over some time period.   John   From: Jason Keirstead <Jason.Keirstead@ca.ibm.com> Date: Thursday, September 1, 2016 at 4:53 PM To: "Wunder, John A." <jwunder@mitre.org> Cc: Bret Jordan <bret.jordan@bluecoat.com>, John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]   Actually... In the case of a network flow they would be expected to be different. Flows have distinct start and end times. -- Sent from my mobile device, please excuse any typos. Wunder, John A. --- Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.] ---   From: "Wunder, John A." <jwunder@mitre.org> To: "Jordan, Bret" <bret.jordan@bluecoat.com> Cc: "John-Mark Gurney" <jmg@newcontext.com>, cti-stix@lists.oasis-open.org Date: Tue, Aug 30, 2016 6:12 PM Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]   The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection.   How do you think it should work?   From: Bret Jordan <bret.jordan@bluecoat.com> Date: Tuesday, August 30, 2016 at 4:56 PM To: "Wunder, John A." <jwunder@mitre.org> Cc: John-Mark Gurney <jmg@newcontext.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> Subject: Re: [cti-stix] [jmg@newcontext.com: Observed Data comments.]   For network-connection though, there are time fields in CybOX land.  So how do those relate to the time stamps in the Observed Data object.???    Thanks,   Bret       Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."    On Aug 30, 2016, at 14:28, Wunder, John A. < jwunder@mitre.org > wrote:   When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor.   On the issues themselves: -             The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. -             I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. John   From:   < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date:   Tuesday, August 30, 2016 at 4:10 PM To:   John-Mark Gurney < jmg@newcontext.com > Cc:   " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject:   Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   How should we address this?  Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot.  Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review.   Thanks,   Bret       Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."    On Aug 30, 2016, at 12:25, John-Mark Gurney < jmg@newcontext.com > wrote:   Did this get lost in the shuffle last week?  I didn't see any discussion on this. ----- Forwarded message from John-Mark Gurney < jmg@newcontext.com > ----- Date: Mon, 22 Aug 2016 17:08:44 -0700 From: John-Mark Gurney < jmg@newcontext.com > To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Observed Data comments. Hello, Sorry for not getting this in sooner, but I have questions/comments about the Observed Data object. There is nothing in the spec that requires first_observed to be equal to last_observed when number_observed is 1. How is a tool who receives such an object to handle this? In the case of when you don't know exactly when the observed data was recorded, the _presision property added to first_observed can convey this information correctly, and most acurately. P.S. IMO, we should also make the number_observed default to 1, and make it optional.  By setting a resonable default, we can reduce the size of objects in common cases. --   John-Mark ----- End forwarded message ----- --   John-Mark --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that   generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php      

Exactly.  In a sandbox you may observe a piece of malware for 10 minutes waiting for it to wake up and do something. Bret  Sent from my Commodore 64 On Sep 1, 2016, at 3:37 PM, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: OK, clarified,  I agree, They should not have to be the same. Discrete observations can have start and end times. Off the top of my head, in addition to Network flows, Process executions (malware analysis) also have a start and end time -- Sent from my mobile device, please excuse any typos. Wunder, John A. --- Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.] --- From: "Wunder, John A." < jwunder@mitre.org > To: "Jason Keirstead" < Jason.Keirstead@ca.ibm.com > Cc: "Jordan, Bret" < bret.jordan@bluecoat.com >, "John-Mark Gurney" < jmg@newcontext.com >, cti-stix@lists.oasis-open.org Date: Thu, Sep 1, 2016 6:20 PM Subject: Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.] I don’t really follow what you’re saying here …we have a bunch of timestamps to consider:   -           Network Connection/Flow Start (time the connection opened) -           Network Connection/Flow End (time the connection closed) -           Observed Data Start (time the observed data was first observed) -           Observed Data End (time the observed data was last observed)   My point was just that Observed Data Start time (time you started collecting the observed data) should match the network connection start time (time the connection opened) and the Observed Data End Time (time you finished collecting the observed data) should match the network connection end time (time the connection closed).   I was suggesting that we should not require the Observed Data Start/End times to be the same when you only have one observation because sometimes single observations (network connections being the best example) occur over some time period.   John   From: Jason Keirstead < Jason.Keirstead@ca.ibm.com > Date: Thursday, September 1, 2016 at 4:53 PM To: "Wunder, John A." < jwunder@mitre.org > Cc: Bret Jordan < bret.jordan@bluecoat.com >, John-Mark Gurney < jmg@newcontext.com >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   Actually... In the case of a network flow they would be expected to be different. Flows have distinct start and end times. -- Sent from my mobile device, please excuse any typos. Wunder, John A. --- Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.] ---   From: "Wunder, John A." < jwunder@mitre.org > To: "Jordan, Bret" < bret.jordan@bluecoat.com > Cc: "John-Mark Gurney" < jmg@newcontext.com >, cti-stix@lists.oasis-open.org Date: Tue, Aug 30, 2016 6:12 PM Subject: Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   The timestamps on Observed Data are the times the observations were made. So if a sensor is “observing” the network connection the time the connection is initiated is the first_observed and the time the connection is closed is the last_observed. So they would likely be the same as the network connection timestamps in the case of a single observation of a network connection.   How do you think it should work?   From: Bret Jordan < bret.jordan@bluecoat.com > Date: Tuesday, August 30, 2016 at 4:56 PM To: "Wunder, John A." < jwunder@mitre.org > Cc: John-Mark Gurney < jmg@newcontext.com >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   For network-connection though, there are time fields in CybOX land.  So how do those relate to the time stamps in the Observed Data object.???    Thanks,   Bret       Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."    On Aug 30, 2016, at 14:28, Wunder, John A. < jwunder@mitre.org > wrote:   When we push a specification for public review we’ll need to reformat it the OASIS templates, and IMO we probably should have another vote to approve that anyway since it has an additional conformance section. So I feel like we could do that reformat, make any changes we identify and explicitly call them out, then vote to approve that. Gary Katz also had some suggestions, though more minor.   On the issues themselves: -             The reason first_observed and last_observed might be a range if the count=1 is for events that are not atomic. A network connection might be open for minutes or even hours, so a point time that it was observed may not make sense. So I don’t think we should have a requirement that they be the same when the count is one. -             I would be fine making count optional and defaulting to 1. I’m also fine as-is, would like to hear from more people about whether we should change this. John   From:   < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date:   Tuesday, August 30, 2016 at 4:10 PM To:   John-Mark Gurney < jmg@newcontext.com > Cc:   " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject:   Re: [cti-stix] [ jmg@newcontext.com : Observed Data comments.]   How should we address this?  Can we do a simple editorial change or is there something more substantial we need to make and then do another ballot.  Keep in mind we can do as many Committee Specification Draft releases we want before we go to a Committee Specification with Public Review.   Thanks,   Bret       Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."    On Aug 30, 2016, at 12:25, John-Mark Gurney < jmg@newcontext.com > wrote:   Did this get lost in the shuffle last week?  I didn't see any discussion on this. ----- Forwarded message from John-Mark Gurney < jmg@newcontext.com > ----- Date: Mon, 22 Aug 2016 17:08:44 -0700 From: John-Mark Gurney < jmg@newcontext.com > To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Observed Data comments. Hello, Sorry for not getting this in sooner, but I have questions/comments about the Observed Data object. There is nothing in the spec that requires first_observed to be equal to last_observed when number_observed is 1. How is a tool who receives such an object to handle this? In the case of when you don't know exactly when the observed data was recorded, the _presision property added to first_observed can convey this information correctly, and most acurately. P.S. IMO, we should also make the number_observed default to 1, and make it optional.  By setting a resonable default, we can reduce the size of objects in common cases. --   John-Mark ----- End forwarded message ----- --   John-Mark --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that   generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php      

Sounds reasonable to me. Cheers Terry MacDonald Cosive On 31/08/2016 6:25 AM, "John-Mark Gurney" < jmg@newcontext.com > wrote: Did this get lost in the shuffle last week?  I didn't see any discussion on this. ----- Forwarded message from John-Mark Gurney < jmg@newcontext.com > ----- Date: Mon, 22 Aug 2016 17:08:44 -0700 From: John-Mark Gurney < jmg@newcontext.com > To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Observed Data comments. Hello, Sorry for not getting this in sooner, but I have questions/comments about the Observed Data object. There is nothing in the spec that requires first_observed to be equal to last_observed when number_observed is 1. How is a tool who receives such an object to handle this? In the case of when you don't know exactly when the observed data was recorded, the _presision property added to first_observed can convey this information correctly, and most acurately. P.S. IMO, we should also make the number_observed default to 1, and make it optional.  By setting a resonable default, we can reduce the size of objects in common cases. -- John-Mark ----- End forwarded message ----- -- John-Mark ------------------------------ ------------------------------ --------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/ apps/org/workgroup/portal/my_ workgroups.php