CTI STIX Subcommittee

 View Only
Expand all | Collapse all

Re: [cti-stix] Deterministic IDs - pas de deux

  • 1.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-04-2016 16:58




    I see us having three options:

    We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion.
    Given those choices, in order of preference I would say: 2, 1, 3. My reasoning:

    We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick one way
    and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing UUID4
    is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and,
    when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts and defining
    a set of fields that you use in every case to match exactly and do correlation might not be the right approach.

    John




    From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org >
    Date: Wednesday, May 4, 2016 at 12:29 PM
    To: Eric Burger < eric.burger@georgetown.edu >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
    Subject: [cti-stix] Deterministic IDs - pas de deux







    @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate.


    Eric,


    Many thanks for engaging in the discourse.  


    Re: "What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they
    need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime."


    I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:


     (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the "random" generator, this is not always the case for Version 4 UUIDs (as described in the RFC).


    (2) Any "random" UUIDs generated by Version 4 are easily discernible in the UUID.


    (3) One can makes inferences from a Version 5 UUID in a "Community of Trust" where Source Namespaces used to "sign" and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID.


    Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee?



    Patrick Maroney
    President
    Integrated Networking Technologies, Inc.
    Desk:
    (856)983-0001
    Cell:
    (609)841-5104
    Email:
    pmaroney@specere.org



    _____________________________
    From: Eric Burger < eric.burger@georgetown.edu >
    Sent: Wednesday, May 4, 2016 11:50 AM
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    To: < cti-stix@lists.oasis-open.org >



    Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack .


    My observations:
    What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used.


    What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they
    need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime.


    Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up.


    Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and
    it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are
    getting a UUID generator for free - you won’t have to think about it.



    On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote:





    re: "So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why"."   



    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:



     (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.     
     
    (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = "." 1*DIGIT]






    The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10
    billion bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations.




    Community members provided real world use cases and challenges where (1) "My" products operate at these speeds, (2) "My" use cases require sub-millisecond Timestamps, etc.    Paraphrasing
    the arguments made against the requested change included (1) "My" product doesn't operate at these speeds, (2) "My" use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds,
    (4) We're not going to pay attention to 'fringe' cases  





    "This is the wrong way to go and here is why":    



      A small faction should not be able to summarily reject a stated community open requirement on the basis of "we don't have or understand this requirement". 









    re: "We have debated this issue almost as long as we have debated timestamps."



    No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the
    concept are not accurate.   Paul Patrick has gotten it right.  


    The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties
    of the Object   .   So for an Object describing the IP Address "1.2.3.4", the  Immutable property of the Object = "1.2.3.4" .    The  "1.2.3.4"  value would NEVER change for this Object.
      Any changes to an objects immutable properties would be a new object, not a new version.


    I'm not going to rehash the rest...


    Bottom Line:



    Current language 










    "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object
    being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID)."






    "This is the wrong way to go and here is why":  
     







    This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID
    to Version 4 (random/pseudo random).


    We "get" that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal
    implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a "5" instead of a "4".






    Proposed Language





    "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object
    being identified or referenced and [uuid] is an RFC 4122 compliant UUID. "





    Vote it up or down...






    Office:   (856)983-0001
    Cell:       (609)841-5104



    <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png>


    President
    Integrated Networking Technologies, Inc.
    PO
    Box 569
    Marlton, NJ 08053







    From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com >
    Date: Tuesday, May 3, 2016 at 3:09 PM
    To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu >
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties






    I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.  


    We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why".  If either the timestamps or IDs that we
    have proposed are wrong, or are going to make life miserable, please speak up and explain why. 










    Maybe our resident academic can chime in here?  


    Thanks,


    Bret











    Bret Jordan CISSP

    Director of Security Architecture and Standards Office of the CTO

    Blue Coat Systems

    PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE

    7415 0050
    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 











    On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote:



    What would be the proposed third value used in the tuple that would not break versioning?

    -
    Jason Keirstead
    STSM, Product Architect, Security Intelligence, IBM Security Systems
    www.ibm.com/security
    www.securityintelligence.com

    Without data, all you are is just another person with an opinion - Unknown


    <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion
    abo

    From: Paul Patrick < ppatrick@isightpartners.com >
    To: "Jordan, Bret" < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org >
    Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
    Date: 05/03/2016 01:37 PM
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    Sent by: < cti-stix@lists.oasis-open.org >





    Folks,

    I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments
    correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email.

    In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly
    the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the
    hash there isn’t the issue of breaking relationships.

    From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support.
    In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the
    UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless.

    So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used.

    I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes.


    Paul Patrick

    From: < cti-stix@lists.oasis-open.org >
    on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com >
    Date: Monday, May 2, 2016 at 6:01 PM
    To: Patrick Maroney < Pmaroney@Specere.org >
    Cc: "Wunder, John A." < jwunder@mitre.org >,
    " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    Resent-From: < Paul.Patrick@FireEye.com >


    Pat,

    Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need
    to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break.

    Thanks,

    Bret



    Bret Jordan CISSP
    Director of Security Architecture and Standards Office of the CTO
    Blue Coat Systems
    PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE

    7415 0050
    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."


    On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote:

    re: . " This is the start of the 2 day review process prior to the initial motion, so please provide
    any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning."


    I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers.
    The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method.

    Format and language are notional and will need corrections by the editors.

    Basis:

    If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation


    -- They can!!!


    if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific
    to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc.



    -- They can To!!!



    The method of generation of any UUID can of course be determined from the generated UUID.



    -- Win-Win for both camps and with no cost that I can discern







    4.5.? Identifier
    Type Name: identifier
    Status: Review
    MVP: Yes

    An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object
    being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

    4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

    4.5.1.?2 Examples
    {
    "type": "indicator",
    "id": "indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836",
    ...
    }

    4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting.

    Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue)

    PublIcNameSpace = NameSpace[.subNameSpace]

    ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace)


    (4.5.2.1) Examples

    NameSpace = ' specere.org '
    subNameSpace = 'isac-2'
    ObjectClass = 'AddressObject'
    ObjectType = 'ipv4-addr'
    ObjectValue = '1.2.3.5'

    AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7

    Demonstrate deterministic generation of CTI Object IDs
    Generates Namespace Prefix deterministic generation of CTI Object IDs

    Type:AddressObjectType.ipv4-addr

    Value:1.2.3.4

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30


    Type:AddressObjectType.ipv4-addr

    Value:1.2.3.5

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7


    Type:DomainNameType.TLD

    Value: badguys.com

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac


    Type:URIObjectType.URL

    Value: http://www.badguys.com/kickme?Hard

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2


    Type:pattern

    Value:
    "pattern": { "type": "twigs", "properties": [
    ],
    "prop1": {
    }, "prop2": {
    }, "prop3": {
    "key":"FileObject:hashes/hash/simple_hash_value", "operator":"equals", "value":"c38862b4835729d979e7940d72a48172"
    "key":"FileObject:file_name", "operator":"contains", "value":"abcd.dll"
    "key":"WinRegistryKeyObject:key",
    "operator":"equals", "value":".DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7"
    }, "prop4": {
    } "prop5": {
    }
    "key":"WinRegistryKeyObject:hive", "operator":"equals", "value":"HKEY_USERS"
    "key":"IPv4AddressObject:hive", "operator":"equals", "value":"1.2.3.4"
    "condition": "(prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS" }


    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: f7634169-2a18-5971-a919-585a4a913b72

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8







    4.1.3. Version

    The version number is in the most significant 4 bits of the time
    stamp (bits 4 through 7 of the time_hi_and_version field).

    The following table lists the currently-defined versions for this
    UUID variant.

    Msb0 Msb1 Msb2 Msb3 Version Description

    0 0 0 1 1 The time-based version
    specified in this document.

    0 0 1 0 2 DCE Security version, with
    embedded POSIX UIDs.


    0 0 1 1 3 The name-based version
    specified in this document
    that uses MD5 hashing.

    0 1 0 0 4 The randomly or pseudo-
    randomly generated version
    specified in this document.

    0 1 0 1 5 The name-based version
    specified in this document
    that uses SHA-1 hashing.

    The version is more accurately a sub-type; again, we retain the term
    for compatibility.


    Patrick Maroney
    Office:
    (856)983-0001
    Cell:
    (609)841-5104

    <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png>

    President
    Integrated Networking Technologies, Inc.
    PO
    Box 569
    Marlton, NJ 08053

    From: < cti-stix@lists.oasis-open.org >
    on behalf of John Wunder < jwunder@mitre.org >
    Date: Friday, April 29, 2016 at 4:23 PM
    To: " cti-stix@lists.oasis-open.org "
    < cti-stix@lists.oasis-open.org >
    Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties

    All,

    As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development
    cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text.

    In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process:


    Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending
    on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the
    draft status (Status = Draft)

    Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development”
    phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward.

    We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic.

    For this first round, please review the following sections in the STIX 2.0 specification:

    Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q
    Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf
    Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j
    Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86

    This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make
    the motions to approve these sections on Wednesday morning.

    Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time.

    John































  • 2.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-04-2016 17:02
    I can live with this. -- Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ > On May 4, 2016 12:58 PM, "Wunder, John A." < jwunder@mitre.org > wrote: I see us having three options: We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion. Given those choices, in order of preference I would say: 2, 1, 3. My reasoning: We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach. John From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org > Date: Wednesday, May 4, 2016 at 12:29 PM To: Eric Burger < eric.burger@georgetown.edu >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Deterministic IDs - pas de deux @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate. Eric, Many thanks for engaging in the discourse.   Re: "What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime." I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:  (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the "random" generator, this is not always the case for Version 4 UUIDs (as described in the RFC). (2) Any "random" UUIDs generated by Version 4 are easily discernible in the UUID. (3) One can makes inferences from a Version 5 UUID in a "Community of Trust" where Source Namespaces used to "sign" and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID. Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee? Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Eric Burger < eric.burger@georgetown.edu > Sent: Wednesday, May 4, 2016 11:50 AM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties To: < cti-stix@lists.oasis-open.org > Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack . My observations: What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used. What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up. Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are getting a UUID generator for free - you won’t have to think about it. On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote: re: "So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why"."    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:  (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.        (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = "." 1*DIGIT] The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10 billion bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations. Community members provided real world use cases and challenges where (1) "My" products operate at these speeds, (2) "My" use cases require sub-millisecond Timestamps, etc.    Paraphrasing the arguments made against the requested change included (1) "My" product doesn't operate at these speeds, (2) "My" use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds, (4) We're not going to pay attention to 'fringe' cases   "This is the wrong way to go and here is why":       A small faction should not be able to summarily reject a stated community open requirement on the basis of "we don't have or understand this requirement".  re: "We have debated this issue almost as long as we have debated timestamps." No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the concept are not accurate.   Paul Patrick has gotten it right.   The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties of the Object   .   So for an Object describing the IP Address "1.2.3.4", the  Immutable property of the Object = "1.2.3.4" .    The  "1.2.3.4"  value would NEVER change for this Object.   Any changes to an objects immutable properties would be a new object, not a new version. I'm not going to rehash the rest... Bottom Line: Current language  "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID)." "This is the wrong way to go and here is why":     This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID to Version 4 (random/pseudo random). We "get" that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a "5" instead of a "4". Proposed Language "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant UUID. " Vote it up or down... Office:   (856)983-0001 Cell:       (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date: Tuesday, May 3, 2016 at 3:09 PM To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.   We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why".  If either the timestamps or IDs that we have proposed are wrong, or are going to make life miserable, please speak up and explain why.  Maybe our resident academic can chime in here?   Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: What would be the proposed third value used in the tuple that would not break versioning? - Jason Keirstead STSM, Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo From: Paul Patrick < ppatrick@isightpartners.com > To: "Jordan, Bret" < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org > Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Date: 05/03/2016 01:37 PM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Sent by: < cti-stix@lists.oasis-open.org > Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email. In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of breaking relationships. From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support. In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless. So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used. I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes. Paul Patrick From: < cti-stix@lists.oasis-open.org > on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com > Date: Monday, May 2, 2016 at 6:01 PM To: Patrick Maroney < Pmaroney@Specere.org > Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Resent-From: < Paul.Patrick@FireEye.com > Pat, Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote: re: . " This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning." I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method. Format and language are notional and will need corrections by the editors. Basis: If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation -- They can!!! if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc. -- They can To!!! The method of generation of any UUID can of course be determined from the generated UUID. -- Win-Win for both camps and with no cost that I can discern 4.5.? Identifier Type Name: identifier Status: Review MVP: Yes An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1.?2 Examples { "type": "indicator", "id": "indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836", ... } 4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting. Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue) PublIcNameSpace = NameSpace[.subNameSpace] ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace) (4.5.2.1) Examples NameSpace = ' specere.org ' subNameSpace = 'isac-2' ObjectClass = 'AddressObject' ObjectType = 'ipv4-addr' ObjectValue = '1.2.3.5' AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7 Demonstrate deterministic generation of CTI Object IDs Generates Namespace Prefix deterministic generation of CTI Object IDs Type:AddressObjectType.ipv4-addr Value:1.2.3.4 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30 Type:AddressObjectType.ipv4-addr Value:1.2.3.5 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7 Type:DomainNameType.TLD Value: badguys.com Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac Type:URIObjectType.URL Value: http://www.badguys.com/kickme?Hard Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2 Type:pattern Value: "pattern": { "type": "twigs", "properties": [ ], "prop1": { }, "prop2": { }, "prop3": { "key":"FileObject:hashes/hash/simple_hash_value", "operator":"equals", "value":"c38862b4835729d979e7940d72a48172" "key":"FileObject:file_name", "operator":"contains", "value":"abcd.dll" "key":"WinRegistryKeyObject:key", "operator":"equals", "value":".DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7" }, "prop4": { } "prop5": { } "key":"WinRegistryKeyObject:hive", "operator":"equals", "value":"HKEY_USERS" "key":"IPv4AddressObject:hive", "operator":"equals", "value":"1.2.3.4" "condition": "(prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS" } Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: f7634169-2a18-5971-a919-585a4a913b72 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8 4.1.3. Version The version number is in the most significant 4 bits of the time stamp (bits 4 through 7 of the time_hi_and_version field). The following table lists the currently-defined versions for this UUID variant. Msb0 Msb1 Msb2 Msb3 Version Description 0 0 0 1 1 The time-based version specified in this document. 0 0 1 0 2 DCE Security version, with embedded POSIX UIDs. 0 0 1 1 3 The name-based version specified in this document that uses MD5 hashing. 0 1 0 0 4 The randomly or pseudo- randomly generated version specified in this document. 0 1 0 1 5 The name-based version specified in this document that uses SHA-1 hashing. The version is more accurately a sub-type; again, we retain the term for compatibility. Patrick Maroney Office: (856)983-0001 Cell: (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of John Wunder < jwunder@mitre.org > Date: Friday, April 29, 2016 at 4:23 PM To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties All, As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text. In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process: Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft) Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward. We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic. For this first round, please review the following sections in the STIX 2.0 specification: Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86 This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time. John


  • 3.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-04-2016 17:14
    I agree with Wunder on all of his points.  If we decide to do deterministic IDs, than I think we will need to redo versioning, for the reasons that John points out, and others that we do not yet fully realize..  And if we are going to back and redo all of these things, than we should probably go back and redo timestamps (there seems to be more people that dislike our timestamps than those that dislike UUIDv4 based IDs).  This would keep parity and fairness in the community.   If we do deterministic IDs, then we will need to determine which fields in each TLO will be used, on a TLO by TLO basis.  We will also need to add a field to each TLO that includes namespace and other things that are used for the deterministic IDs. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 4, 2016, at 11:01, Eric Burger < Eric.Burger@georgetown.edu > wrote: I can live with this. -- Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ > On May 4, 2016 12:58 PM, Wunder, John A. < jwunder@mitre.org > wrote: I see us having three options: We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion. Given those choices, in order of preference I would say: 2, 1, 3. My reasoning: We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach. John From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org > Date: Wednesday, May 4, 2016 at 12:29 PM To: Eric Burger < eric.burger@georgetown.edu >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Deterministic IDs - pas de deux @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate. Eric, Many thanks for engaging in the discourse.   Re: What I would also observe is the sentiment that we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future is flawed. Unless people who do not think they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:  (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the random generator, this is not always the case for Version 4 UUIDs (as described in the RFC). (2) Any random UUIDs generated by Version 4 are easily discernible in the UUID. (3) One can makes inferences from a Version 5 UUID in a Community of Trust where Source Namespaces used to sign and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID. Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee? Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Eric Burger < eric.burger@georgetown.edu > Sent: Wednesday, May 4, 2016 11:50 AM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties To: < cti-stix@lists.oasis-open.org > Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack . My observations: What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used. What I would also observe is the sentiment that we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future is flawed. Unless people who do not think they need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up. Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are getting a UUID generator for free - you won’t have to think about it. On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote: re: So, as with timestamps, there is a difference between I just do not like it and this is the wrong way to go and here is why .    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:  (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.        (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = . 1*DIGIT] The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10 billion bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations. Community members provided real world use cases and challenges where (1) My products operate at these speeds, (2) My use cases require sub-millisecond Timestamps, etc.    Paraphrasing the arguments made against the requested change included (1) My product doesn't operate at these speeds, (2) My use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds, (4) We're not going to pay attention to 'fringe' cases   This is the wrong way to go and here is why :       A small faction should not be able to summarily reject a stated community open requirement on the basis of we don't have or understand this requirement .  re: We have debated this issue almost as long as we have debated timestamps. No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the concept are not accurate.   Paul Patrick has gotten it right.   The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties of the Object   .   So for an Object describing the IP Address 1.2.3.4 , the  Immutable property of the Object = 1.2.3.4 .    The  1.2.3.4   value would NEVER change for this Object.   Any changes to an objects immutable properties would be a new object, not a new version. I'm not going to rehash the rest... Bottom Line: Current language  An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). This is the wrong way to go and here is why :     This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID to Version 4 (random/pseudo random). We get that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a 5 instead of a 4 . Proposed Language An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant UUID. Vote it up or down... Office:   (856)983-0001 Cell:       (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date: Tuesday, May 3, 2016 at 3:09 PM To: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.   We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between I just do not like it and this is the wrong way to go and here is why .  If either the timestamps or IDs that we have proposed are wrong, or are going to make life miserable, please speak up and explain why.  Maybe our resident academic can chime in here?   Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: What would be the proposed third value used in the tuple that would not break versioning? - Jason Keirstead STSM, Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo From: Paul Patrick < ppatrick@isightpartners.com > To: Jordan, Bret < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org > Cc: Wunder, John A. < jwunder@mitre.org >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Date: 05/03/2016 01:37 PM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Sent by: < cti-stix@lists.oasis-open.org > Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email. In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of breaking relationships. From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support. In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless. So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used. I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes. Paul Patrick From: < cti-stix@lists.oasis-open.org > on behalf of Jordan, Bret < bret.jordan@bluecoat.com > Date: Monday, May 2, 2016 at 6:01 PM To: Patrick Maroney < Pmaroney@Specere.org > Cc: Wunder, John A. < jwunder@mitre.org >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Resent-From: < Paul.Patrick@FireEye.com > Pat, Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg. On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote: re: . This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method. Format and language are notional and will need corrections by the editors. Basis: If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation -- They can!!! if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc. -- They can To!!! The method of generation of any UUID can of course be determined from the generated UUID. -- Win-Win for both camps and with no cost that I can discern 4.5.? Identifier Type Name: identifier Status: Review MVP: Yes An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1.?2 Examples { type : indicator , id : indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836 , ... } 4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting. Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue) PublIcNameSpace = NameSpace[.subNameSpace] ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace) (4.5.2.1) Examples NameSpace = ' specere.org ' subNameSpace = 'isac-2' ObjectClass = 'AddressObject' ObjectType = 'ipv4-addr' ObjectValue = '1.2.3.5' AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7 Demonstrate deterministic generation of CTI Object IDs Generates Namespace Prefix deterministic generation of CTI Object IDs Type:AddressObjectType.ipv4-addr Value:1.2.3.4 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30 Type:AddressObjectType.ipv4-addr Value:1.2.3.5 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7 Type:DomainNameType.TLD Value: badguys.com Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac Type:URIObjectType.URL Value: http://www.badguys.com/kickme?Hard Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2 Type:pattern Value: pattern : { type : twigs , properties : [ ], prop1 : { }, prop2 : { }, prop3 : { key : FileObject:hashes/hash/simple_hash_value , operator : equals , value : c38862b4835729d979e7940d72a48172 key : FileObject:file_name , operator : contains , value : abcd.dll key : WinRegistryKeyObject:key , operator : equals , value : .DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7 }, prop4 : { } prop5 : { } key : WinRegistryKeyObject:hive , operator : equals , value : HKEY_USERS key : IPv4AddressObject:hive , operator : equals , value : 1.2.3.4 condition : (prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS } Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: f7634169-2a18-5971-a919-585a4a913b72 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8 4.1.3. Version The version number is in the most significant 4 bits of the time stamp (bits 4 through 7 of the time_hi_and_version field). The following table lists the currently-defined versions for this UUID variant. Msb0 Msb1 Msb2 Msb3 Version Description 0 0 0 1 1 The time-based version specified in this document. 0 0 1 0 2 DCE Security version, with embedded POSIX UIDs. 0 0 1 1 3 The name-based version specified in this document that uses MD5 hashing. 0 1 0 0 4 The randomly or pseudo- randomly generated version specified in this document. 0 1 0 1 5 The name-based version specified in this document that uses SHA-1 hashing. The version is more accurately a sub-type; again, we retain the term for compatibility. Patrick Maroney Office: (856)983-0001 Cell: (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of John Wunder < jwunder@mitre.org > Date: Friday, April 29, 2016 at 4:23 PM To: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties All, As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text. In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process: Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft) Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward. We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic. For this first round, please review the following sections in the STIX 2.0 specification: Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86 This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time. John Attachment: signature.asc Description: Message signed with OpenPGP using GPGMail


  • 4.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-04-2016 17:44




    Why not have a required identifier based on UUID4 (my preference) and an if vendors want a deterministic id then they can include an optional attribute for that. For systems that want to use that optional identifier it would provide support for.


    allan












    From: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com >
    Date: Wednesday, May 4, 2016 at 10:13 AM
    To: Eric Burger < Eric.Burger@georgetown.edu >
    Cc: "Wunder, John" < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
    Subject: Re: [cti-stix] Deterministic IDs - pas de deux






    I agree with Wunder on all of his points. 


    If we decide to do deterministic IDs, than I think we will need to redo versioning, for the reasons that John points out, and others that we do not yet fully realize..  And if we are going to back and redo all of these things, than we should probably
    go back and redo timestamps (there seems to be more people that dislike our timestamps than those that dislike UUIDv4 based IDs).  This would keep parity and fairness in the community.  


    If we do deterministic IDs, then we will need to determine which fields in each TLO will be used, on a TLO by TLO basis.  We will also need to add a field to each TLO that includes namespace and other things that are used for the deterministic
    IDs.











    Thanks,


    Bret











    Bret Jordan CISSP

    Director of Security Architecture and Standards Office of the CTO

    Blue Coat Systems

    PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050
    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 











    On May 4, 2016, at 11:01, Eric Burger < Eric.Burger@georgetown.edu > wrote:


    I can live with this.
    --
    Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ >
    On May 4, 2016 12:58 PM, "Wunder, John A." < jwunder@mitre.org > wrote:



    I see us having three options:

    We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion.
    Given those choices, in order of preference I would say: 2, 1, 3. My reasoning:

    We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick
    one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing
    UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change
    and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts
    and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach.

    John




    From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org >
    Date: Wednesday, May 4, 2016 at 12:29 PM
    To: Eric Burger < eric.burger@georgetown.edu >, " cti-stix@lists.oasis-open.org "
    < cti-stix@lists.oasis-open.org >
    Subject: [cti-stix] Deterministic IDs - pas de deux







    @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate.


    Eric,


    Many thanks for engaging in the discourse.  


    Re: "What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think
    they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime."


    I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:


     (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the "random" generator, this is not always the case for Version 4 UUIDs (as described in the RFC).


    (2) Any "random" UUIDs generated by Version 4 are easily discernible in the UUID.


    (3) One can makes inferences from a Version 5 UUID in a "Community of Trust" where Source Namespaces used to "sign" and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID.


    Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee?



    Patrick Maroney
    President
    Integrated Networking Technologies, Inc.
    Desk: (856)983-0001
    Cell: (609)841-5104
    Email: pmaroney@specere.org



    _____________________________
    From: Eric Burger < eric.burger@georgetown.edu >
    Sent: Wednesday, May 4, 2016 11:50 AM
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    To: < cti-stix@lists.oasis-open.org >


    Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack .


    My observations:
    What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used.


    What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they
    need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime.


    Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up.


    Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and
    it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are
    getting a UUID generator for free - you won’t have to think about it.



    On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote:




    re: "So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why"."   



    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:



     (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.     
     
    (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = "." 1*DIGIT]






    The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10 billion
    bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations.




    Community members provided real world use cases and challenges where (1) "My" products operate at these speeds, (2) "My" use cases require sub-millisecond Timestamps, etc.    Paraphrasing
    the arguments made against the requested change included (1) "My" product doesn't operate at these speeds, (2) "My" use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds,
    (4) We're not going to pay attention to 'fringe' cases  





    "This is the wrong way to go and here is why":    



      A small faction should not be able to summarily reject a stated community open requirement on the basis of "we don't have or understand this requirement". 









    re: "We have debated this issue almost as long as we have debated timestamps."



    No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the concept
    are not accurate.   Paul Patrick has gotten it right.  


    The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties of
    the Object   .   So for an Object describing the IP Address "1.2.3.4", the  Immutable property of the Object = "1.2.3.4" .    The  "1.2.3.4"  value would NEVER change for this Object.
      Any changes to an objects immutable properties would be a new object, not a new version.


    I'm not going to rehash the rest...


    Bottom Line:



    Current language 










    "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object
    being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID)."






    "This is the wrong way to go and here is why":  
     







    This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID
    to Version 4 (random/pseudo random).


    We "get" that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal
    implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a "5" instead of a "4".






    Proposed Language





    "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object
    being identified or referenced and [uuid] is an RFC 4122 compliant UUID. "





    Vote it up or down...






    Office:   (856)983-0001
    Cell:       (609)841-5104



    <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png>


    President
    Integrated Networking Technologies, Inc.
    PO
    Box 569
    Marlton, NJ 08053







    From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com >
    Date: Tuesday, May 3, 2016 at 3:09 PM
    To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >,
    Eric Burger < Eric.Burger@georgetown.edu >
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties





    I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.  


    We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why".  If either the timestamps or IDs that we
    have proposed are wrong, or are going to make life miserable, please speak up and explain why. 









    Maybe our resident academic can chime in here?  


    Thanks,


    Bret











    Bret Jordan CISSP

    Director of Security Architecture and Standards Office of the CTO

    Blue Coat Systems

    PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE
    7415 0050
    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 











    On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote:



    What would be the proposed third value used in the tuple that would not break versioning?

    -
    Jason Keirstead
    STSM, Product Architect, Security Intelligence, IBM Security Systems
    www.ibm.com/security
    www.securityintelligence.com

    Without data, all you are is just another person with an opinion - Unknown


    <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo

    From: Paul Patrick < ppatrick@isightpartners.com >
    To: "Jordan, Bret" < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org >
    Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org "
    < cti-stix@lists.oasis-open.org >
    Date: 05/03/2016 01:37 PM
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    Sent by: < cti-stix@lists.oasis-open.org >





    Folks,

    I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments
    correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email.

    In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly
    the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the
    hash there isn’t the issue of breaking relationships.

    From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support.
    In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the
    UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless.

    So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used.

    I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes.


    Paul Patrick

    From: < cti-stix@lists.oasis-open.org >
    on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com >
    Date: Monday, May 2, 2016 at 6:01 PM
    To: Patrick Maroney < Pmaroney@Specere.org >
    Cc: "Wunder, John A." < jwunder@mitre.org >,
    " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    Resent-From: < Paul.Patrick@FireEye.com >


    Pat,

    Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need
    to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break.

    Thanks,

    Bret



    Bret Jordan CISSP
    Director of Security Architecture and Standards Office of the CTO
    Blue Coat Systems
    PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE

    7415 0050
    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."


    On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org >
    wrote:

    re: . " This is the start of the 2 day review process prior to the initial motion, so please provide
    any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning."


    I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers.
    The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method.

    Format and language are notional and will need corrections by the editors.

    Basis:

    If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation


    -- They can!!!


    if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific
    to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc.



    -- They can To!!!



    The method of generation of any UUID can of course be determined from the generated UUID.



    -- Win-Win for both camps and with no cost that I can discern







    4.5.? Identifier
    Type Name: identifier
    Status: Review
    MVP: Yes

    An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object
    being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

    4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

    4.5.1.?2 Examples
    {
    "type": "indicator",
    "id": "indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836",
    ...
    }

    4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting.

    Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue)

    PublIcNameSpace = NameSpace[.subNameSpace]

    ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace)


    (4.5.2.1) Examples

    NameSpace = ' specere.org '
    subNameSpace = 'isac-2'
    ObjectClass = 'AddressObject'
    ObjectType = 'ipv4-addr'
    ObjectValue = '1.2.3.5'

    AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7

    Demonstrate deterministic generation of CTI Object IDs
    Generates Namespace Prefix deterministic generation of CTI Object IDs

    Type:AddressObjectType.ipv4-addr

    Value:1.2.3.4

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30


    Type:AddressObjectType.ipv4-addr

    Value:1.2.3.5

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7


    Type:DomainNameType.TLD

    Value: badguys.com

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac


    Type:URIObjectType.URL

    Value: http://www.badguys.com/kickme?Hard

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2


    Type:pattern

    Value:
    "pattern": { "type": "twigs", "properties": [
    ],
    "prop1": {
    }, "prop2": {
    }, "prop3": {
    "key":"FileObject:hashes/hash/simple_hash_value", "operator":"equals", "value":"c38862b4835729d979e7940d72a48172"
    "key":"FileObject:file_name", "operator":"contains", "value":"abcd.dll"
    "key":"WinRegistryKeyObject:key",
    "operator":"equals", "value":".DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7"
    }, "prop4": {
    } "prop5": {
    }
    "key":"WinRegistryKeyObject:hive", "operator":"equals", "value":"HKEY_USERS"
    "key":"IPv4AddressObject:hive", "operator":"equals", "value":"1.2.3.4"
    "condition": "(prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS" }


    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: f7634169-2a18-5971-a919-585a4a913b72

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8







    4.1.3. Version

    The version number is in the most significant 4 bits of the time
    stamp (bits 4 through 7 of the time_hi_and_version field).

    The following table lists the currently-defined versions for this
    UUID variant.

    Msb0 Msb1 Msb2 Msb3 Version Description

    0 0 0 1 1 The time-based version
    specified in this document.

    0 0 1 0 2 DCE Security version, with
    embedded POSIX UIDs.


    0 0 1 1 3 The name-based version
    specified in this document
    that uses MD5 hashing.

    0 1 0 0 4 The randomly or pseudo-
    randomly generated version
    specified in this document.

    0 1 0 1 5 The name-based version
    specified in this document
    that uses SHA-1 hashing.

    The version is more accurately a sub-type; again, we retain the term
    for compatibility.


    Patrick Maroney
    Office:
    (856)983-0001
    Cell:
    (609)841-5104

    <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png>

    President
    Integrated Networking Technologies, Inc.
    PO
    Box 569
    Marlton, NJ 08053

    From: < cti-stix@lists.oasis-open.org >
    on behalf of John Wunder < jwunder@mitre.org >
    Date: Friday, April 29, 2016 at 4:23 PM
    To: " cti-stix@lists.oasis-open.org "
    < cti-stix@lists.oasis-open.org >
    Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties

    All,

    As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development
    cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text.

    In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process:


    Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending
    on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the
    draft status (Status = Draft)

    Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development”
    phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward.

    We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic.

    For this first round, please review the following sections in the STIX 2.0 specification:

    Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q
    Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf
    Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j
    Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86

    This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make
    the motions to approve these sections on Wednesday morning.

    Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time.

    John









































  • 5.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-04-2016 17:53
    Re: "Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash." Again this is a mis-characterization of my proposal.  The proposed approach is to use the Immutable properties of an Object.  Any changes to these immutable properties are by definition a new object, not a revision.  Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Allan Thomson < athomson@lookingglasscyber.com > Sent: Wednesday, May 4, 2016 1:44 PM Subject: Re: [cti-stix] Deterministic IDs - pas de deux To: Jordan, Bret < bret.jordan@bluecoat.com >, Eric Burger < eric.burger@georgetown.edu > Cc: < cti-stix@lists.oasis-open.org >, John A. Wunder < jwunder@mitre.org > Why not have a required identifier based on UUID4 (my preference) and an if vendors want a deterministic id then they can include an optional attribute for that. For systems that want to use that optional identifier it would provide support for. allan From: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com > Date: Wednesday, May 4, 2016 at 10:13 AM To: Eric Burger < Eric.Burger@georgetown.edu > Cc: "Wunder, John" < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Deterministic IDs - pas de deux I agree with Wunder on all of his points.  If we decide to do deterministic IDs, than I think we will need to redo versioning, for the reasons that John points out, and others that we do not yet fully realize..  And if we are going to back and redo all of these things, than we should probably go back and redo timestamps (there seems to be more people that dislike our timestamps than those that dislike UUIDv4 based IDs).  This would keep parity and fairness in the community.   If we do deterministic IDs, then we will need to determine which fields in each TLO will be used, on a TLO by TLO basis.  We will also need to add a field to each TLO that includes namespace and other things that are used for the deterministic IDs. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 4, 2016, at 11:01, Eric Burger < Eric.Burger@georgetown.edu > wrote: I can live with this. -- Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ > On May 4, 2016 12:58 PM, "Wunder, John A." < jwunder@mitre.org > wrote: I see us having three options: We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion. Given those choices, in order of preference I would say: 2, 1, 3. My reasoning: We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach. John From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org > Date: Wednesday, May 4, 2016 at 12:29 PM To: Eric Burger < eric.burger@georgetown.edu >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Deterministic IDs - pas de deux @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate. Eric, Many thanks for engaging in the discourse.   Re: "What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime." I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:  (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the "random" generator, this is not always the case for Version 4 UUIDs (as described in the RFC). (2) Any "random" UUIDs generated by Version 4 are easily discernible in the UUID. (3) One can makes inferences from a Version 5 UUID in a "Community of Trust" where Source Namespaces used to "sign" and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID. Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee? Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Eric Burger < eric.burger@georgetown.edu > Sent: Wednesday, May 4, 2016 11:50 AM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties To: < cti-stix@lists.oasis-open.org > Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack . My observations: What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used. What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up. Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are getting a UUID generator for free - you won’t have to think about it. On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote: re: "So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why"."    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:  (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.        (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = "." 1*DIGIT] The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10 billion bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations. Community members provided real world use cases and challenges where (1) "My" products operate at these speeds, (2) "My" use cases require sub-millisecond Timestamps, etc.    Paraphrasing the arguments made against the requested change included (1) "My" product doesn't operate at these speeds, (2) "My" use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds, (4) We're not going to pay attention to 'fringe' cases   "This is the wrong way to go and here is why":       A small faction should not be able to summarily reject a stated community open requirement on the basis of "we don't have or understand this requirement".  re: "We have debated this issue almost as long as we have debated timestamps." No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the concept are not accurate.   Paul Patrick has gotten it right.   The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties of the Object   .   So for an Object describing the IP Address "1.2.3.4", the  Immutable property of the Object = "1.2.3.4" .    The  "1.2.3.4"  value would NEVER change for this Object.   Any changes to an objects immutable properties would be a new object, not a new version. I'm not going to rehash the rest... Bottom Line: Current language  "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID)." "This is the wrong way to go and here is why":     This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID to Version 4 (random/pseudo random). We "get" that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a "5" instead of a "4". Proposed Language "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant UUID. " Vote it up or down... Office:   (856)983-0001 Cell:       (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date: Tuesday, May 3, 2016 at 3:09 PM To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.   We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why".  If either the timestamps or IDs that we have proposed are wrong, or are going to make life miserable, please speak up and explain why.  Maybe our resident academic can chime in here?   Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: What would be the proposed third value used in the tuple that would not break versioning? - Jason Keirstead STSM, Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo From: Paul Patrick < ppatrick@isightpartners.com > To: "Jordan, Bret" < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org > Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Date: 05/03/2016 01:37 PM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Sent by: < cti-stix@lists.oasis-open.org > Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email. In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of breaking relationships. From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support. In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless. So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used. I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes. Paul Patrick From: < cti-stix@lists.oasis-open.org > on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com > Date: Monday, May 2, 2016 at 6:01 PM To: Patrick Maroney < Pmaroney@Specere.org > Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Resent-From: < Paul.Patrick@FireEye.com > Pat, Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote: re: . " This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning." I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method. Format and language are notional and will need corrections by the editors. Basis: If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation -- They can!!! if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc. -- They can To!!! The method of generation of any UUID can of course be determined from the generated UUID. -- Win-Win for both camps and with no cost that I can discern 4.5.? Identifier Type Name: identifier Status: Review MVP: Yes An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1.?2 Examples { "type": "indicator", "id": "indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836", ... } 4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting. Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue) PublIcNameSpace = NameSpace[.subNameSpace] ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace) (4.5.2.1) Examples NameSpace = ' specere.org ' subNameSpace = 'isac-2' ObjectClass = 'AddressObject' ObjectType = 'ipv4-addr' ObjectValue = '1.2.3.5' AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7 Demonstrate deterministic generation of CTI Object IDs Generates Namespace Prefix deterministic generation of CTI Object IDs Type:AddressObjectType.ipv4-addr Value:1.2.3.4 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30 Type:AddressObjectType.ipv4-addr Value:1.2.3.5 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7 Type:DomainNameType.TLD Value: badguys.com Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac Type:URIObjectType.URL Value: http://www.badguys.com/kickme?Hard Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2 Type:pattern Value: "pattern": { "type": "twigs", "properties": [ ], "prop1": { }, "prop2": { }, "prop3": { "key":"FileObject:hashes/hash/simple_hash_value", "operator":"equals", "value":"c38862b4835729d979e7940d72a48172" "key":"FileObject:file_name", "operator":"contains", "value":"abcd.dll" "key":"WinRegistryKeyObject:key", "operator":"equals", "value":".DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7" }, "prop4": { } "prop5": { } "key":"WinRegistryKeyObject:hive", "operator":"equals", "value":"HKEY_USERS" "key":"IPv4AddressObject:hive", "operator":"equals", "value":"1.2.3.4" "condition": "(prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS" } Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: f7634169-2a18-5971-a919-585a4a913b72 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8 4.1.3. Version The version number is in the most significant 4 bits of the time stamp (bits 4 through 7 of the time_hi_and_version field). The following table lists the currently-defined versions for this UUID variant. Msb0 Msb1 Msb2 Msb3 Version Description 0 0 0 1 1 The time-based version specified in this document. 0 0 1 0 2 DCE Security version, with embedded POSIX UIDs. 0 0 1 1 3 The name-based version specified in this document that uses MD5 hashing. 0 1 0 0 4 The randomly or pseudo- randomly generated version specified in this document. 0 1 0 1 5 The name-based version specified in this document that uses SHA-1 hashing. The version is more accurately a sub-type; again, we retain the term for compatibility. Patrick Maroney Office: (856)983-0001 Cell: (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of John Wunder < jwunder@mitre.org > Date: Friday, April 29, 2016 at 4:23 PM To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties All, As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text. In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process: Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft) Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward. We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic. For this first round, please review the following sections in the STIX 2.0 specification: Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86 This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time. John


  • 6.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-04-2016 18:10
    In CybOX this could work... But not really in STIX... We would need to try and identify which fields on each TLO would be considered immutable...  Lets take indicator for example: We have the following fields: title id created_by_ref created_time revision modified_time revoked revision_comment confidence (reserved) title description pattern labels start_time end_time severity (reserved) decay_rate The first 9 fields are not guaranteed to be unique.   Fields 10-17 can all have minor adjustments per the versioning method the community has decided on without getting a new ID... So I am curious to know which fields in the Indicator would be used to form a deterministic ID? Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 4, 2016, at 11:52, Patrick Maroney < Pmaroney@Specere.org > wrote: Re: Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. Again this is a mis-characterization of my proposal.  The proposed approach is to use the Immutable properties of an Object.  Any changes to these immutable properties are by definition a new object, not a revision.  Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Allan Thomson < athomson@lookingglasscyber.com > Sent: Wednesday, May 4, 2016 1:44 PM Subject: Re: [cti-stix] Deterministic IDs - pas de deux To: Jordan, Bret < bret.jordan@bluecoat.com >, Eric Burger < eric.burger@georgetown.edu > Cc: < cti-stix@lists.oasis-open.org >, John A. Wunder < jwunder@mitre.org > Why not have a required identifier based on UUID4 (my preference) and an if vendors want a deterministic id then they can include an optional attribute for that. For systems that want to use that optional identifier it would provide support for. allan From: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > on behalf of Jordan, Bret < bret.jordan@bluecoat.com > Date: Wednesday, May 4, 2016 at 10:13 AM To: Eric Burger < Eric.Burger@georgetown.edu > Cc: Wunder, John < jwunder@mitre.org >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Deterministic IDs - pas de deux I agree with Wunder on all of his points.  If we decide to do deterministic IDs, than I think we will need to redo versioning, for the reasons that John points out, and others that we do not yet fully realize..  And if we are going to back and redo all of these things, than we should probably go back and redo timestamps (there seems to be more people that dislike our timestamps than those that dislike UUIDv4 based IDs).  This would keep parity and fairness in the community.   If we do deterministic IDs, then we will need to determine which fields in each TLO will be used, on a TLO by TLO basis.  We will also need to add a field to each TLO that includes namespace and other things that are used for the deterministic IDs. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 4, 2016, at 11:01, Eric Burger < Eric.Burger@georgetown.edu > wrote: I can live with this. -- Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ > On May 4, 2016 12:58 PM, Wunder, John A. < jwunder@mitre.org > wrote: I see us having three options: We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion. Given those choices, in order of preference I would say: 2, 1, 3. My reasoning: We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach. John From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org > Date: Wednesday, May 4, 2016 at 12:29 PM To: Eric Burger < eric.burger@georgetown.edu >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Deterministic IDs - pas de deux @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate. Eric, Many thanks for engaging in the discourse.   Re: What I would also observe is the sentiment that we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future is flawed. Unless people who do not think they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:  (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the random generator, this is not always the case for Version 4 UUIDs (as described in the RFC). (2) Any random UUIDs generated by Version 4 are easily discernible in the UUID. (3) One can makes inferences from a Version 5 UUID in a Community of Trust where Source Namespaces used to sign and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID. Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee? Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Eric Burger < eric.burger@georgetown.edu > Sent: Wednesday, May 4, 2016 11:50 AM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties To: < cti-stix@lists.oasis-open.org > Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack . My observations: What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used. What I would also observe is the sentiment that we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future is flawed. Unless people who do not think they need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up. Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are getting a UUID generator for free - you won’t have to think about it. On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote: re: So, as with timestamps, there is a difference between I just do not like it and this is the wrong way to go and here is why .    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:  (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.        (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = . 1*DIGIT] The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10 billion bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations. Community members provided real world use cases and challenges where (1) My products operate at these speeds, (2) My use cases require sub-millisecond Timestamps, etc.    Paraphrasing the arguments made against the requested change included (1) My product doesn't operate at these speeds, (2) My use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds, (4) We're not going to pay attention to 'fringe' cases   This is the wrong way to go and here is why :       A small faction should not be able to summarily reject a stated community open requirement on the basis of we don't have or understand this requirement .  re: We have debated this issue almost as long as we have debated timestamps. No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the concept are not accurate.   Paul Patrick has gotten it right.   The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties of the Object   .   So for an Object describing the IP Address 1.2.3.4 , the  Immutable property of the Object = 1.2.3.4 .    The  1.2.3.4   value would NEVER change for this Object.   Any changes to an objects immutable properties would be a new object, not a new version. I'm not going to rehash the rest... Bottom Line: Current language  An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). This is the wrong way to go and here is why :     This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID to Version 4 (random/pseudo random). We get that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a 5 instead of a 4 . Proposed Language An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant UUID. Vote it up or down... Office:   (856)983-0001 Cell:       (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date: Tuesday, May 3, 2016 at 3:09 PM To: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.   We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between I just do not like it and this is the wrong way to go and here is why .  If either the timestamps or IDs that we have proposed are wrong, or are going to make life miserable, please speak up and explain why.  Maybe our resident academic can chime in here?   Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: What would be the proposed third value used in the tuple that would not break versioning? - Jason Keirstead STSM, Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo From: Paul Patrick < ppatrick@isightpartners.com > To: Jordan, Bret < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org > Cc: Wunder, John A. < jwunder@mitre.org >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Date: 05/03/2016 01:37 PM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Sent by: < cti-stix@lists.oasis-open.org > Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email. In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of breaking relationships. From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support. In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless. So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used. I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes. Paul Patrick From: < cti-stix@lists.oasis-open.org > on behalf of Jordan, Bret < bret.jordan@bluecoat.com > Date: Monday, May 2, 2016 at 6:01 PM To: Patrick Maroney < Pmaroney@Specere.org > Cc: Wunder, John A. < jwunder@mitre.org >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Resent-From: < Paul.Patrick@FireEye.com > Pat, Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg. On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote: re: . This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method. Format and language are notional and will need corrections by the editors. Basis: If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation -- They can!!! if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc. -- They can To!!! The method of generation of any UUID can of course be determined from the generated UUID. -- Win-Win for both camps and with no cost that I can discern 4.5.? Identifier Type Name: identifier Status: Review MVP: Yes An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1.?2 Examples { type : indicator , id : indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836 , ... } 4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting. Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue) PublIcNameSpace = NameSpace[.subNameSpace] ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace) (4.5.2.1) Examples NameSpace = ' specere.org ' subNameSpace = 'isac-2' ObjectClass = 'AddressObject' ObjectType = 'ipv4-addr' ObjectValue = '1.2.3.5' AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7 Demonstrate deterministic generation of CTI Object IDs Generates Namespace Prefix deterministic generation of CTI Object IDs Type:AddressObjectType.ipv4-addr Value:1.2.3.4 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30 Type:AddressObjectType.ipv4-addr Value:1.2.3.5 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7 Type:DomainNameType.TLD Value: badguys.com Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac Type:URIObjectType.URL Value: http://www.badguys.com/kickme?Hard Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2 Type:pattern Value: pattern : { type : twigs , properties : [ ], prop1 : { }, prop2 : { }, prop3 : { key : FileObject:hashes/hash/simple_hash_value , operator : equals , value : c38862b4835729d979e7940d72a48172 key : FileObject:file_name , operator : contains , value : abcd.dll key : WinRegistryKeyObject:key , operator : equals , value : .DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7 }, prop4 : { } prop5 : { } key : WinRegistryKeyObject:hive , operator : equals , value : HKEY_USERS key : IPv4AddressObject:hive , operator : equals , value : 1.2.3.4 condition : (prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS } Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: f7634169-2a18-5971-a919-585a4a913b72 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8 4.1.3. Version The version number is in the most significant 4 bits of the time stamp (bits 4 through 7 of the time_hi_and_version field). The following table lists the currently-defined versions for this UUID variant. Msb0 Msb1 Msb2 Msb3 Version Description 0 0 0 1 1 The time-based version specified in this document. 0 0 1 0 2 DCE Security version, with embedded POSIX UIDs. 0 0 1 1 3 The name-based version specified in this document that uses MD5 hashing. 0 1 0 0 4 The randomly or pseudo- randomly generated version specified in this document. 0 1 0 1 5 The name-based version specified in this document that uses SHA-1 hashing. The version is more accurately a sub-type; again, we retain the term for compatibility. Patrick Maroney Office: (856)983-0001 Cell: (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of John Wunder < jwunder@mitre.org > Date: Friday, April 29, 2016 at 4:23 PM To: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties All, As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text. In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process: Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft) Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward. We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic. For this first round, please review the following sections in the STIX 2.0 specification: Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86 This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time. John Attachment: signature.asc Description: Message signed with OpenPGP using GPGMail


  • 7.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-04-2016 18:13
    Sorry minor type'o in the field list (field 1 should be type)... Fixed below.  But as you can see, deterministic IDs will NOT work with STIX. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 4, 2016, at 12:11, Jordan, Bret < bret.jordan@BLUECOAT.COM > wrote: In CybOX this could work... But not really in STIX... We would need to try and identify which fields on each TLO would be considered immutable...  Lets take indicator for example: We have the following fields: type id created_by_ref created_time revision modified_time revoked revision_comment confidence (reserved) title description pattern labels start_time end_time severity (reserved) decay_rate The first 9 fields are not guaranteed to be unique.   Fields 10-17 can all have minor adjustments per the versioning method the community has decided on without getting a new ID... So I am curious to know which fields in the Indicator would be used to form a deterministic ID? Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 4, 2016, at 11:52, Patrick Maroney < Pmaroney@Specere.org > wrote: Re: Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. Again this is a mis-characterization of my proposal.  The proposed approach is to use the Immutable properties of an Object.  Any changes to these immutable properties are by definition a new object, not a revision.  Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Allan Thomson < athomson@lookingglasscyber.com > Sent: Wednesday, May 4, 2016 1:44 PM Subject: Re: [cti-stix] Deterministic IDs - pas de deux To: Jordan, Bret < bret.jordan@bluecoat.com >, Eric Burger < eric.burger@georgetown.edu > Cc: < cti-stix@lists.oasis-open.org >, John A. Wunder < jwunder@mitre.org > Why not have a required identifier based on UUID4 (my preference) and an if vendors want a deterministic id then they can include an optional attribute for that. For systems that want to use that optional identifier it would provide support for. allan From: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > on behalf of Jordan, Bret < bret.jordan@bluecoat.com > Date: Wednesday, May 4, 2016 at 10:13 AM To: Eric Burger < Eric.Burger@georgetown.edu > Cc: Wunder, John < jwunder@mitre.org >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Deterministic IDs - pas de deux I agree with Wunder on all of his points.  If we decide to do deterministic IDs, than I think we will need to redo versioning, for the reasons that John points out, and others that we do not yet fully realize..  And if we are going to back and redo all of these things, than we should probably go back and redo timestamps (there seems to be more people that dislike our timestamps than those that dislike UUIDv4 based IDs).  This would keep parity and fairness in the community.   If we do deterministic IDs, then we will need to determine which fields in each TLO will be used, on a TLO by TLO basis.  We will also need to add a field to each TLO that includes namespace and other things that are used for the deterministic IDs. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 4, 2016, at 11:01, Eric Burger < Eric.Burger@georgetown.edu > wrote: I can live with this. -- Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ > On May 4, 2016 12:58 PM, Wunder, John A. < jwunder@mitre.org > wrote: I see us having three options: We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion. Given those choices, in order of preference I would say: 2, 1, 3. My reasoning: We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach. John From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org > Date: Wednesday, May 4, 2016 at 12:29 PM To: Eric Burger < eric.burger@georgetown.edu >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Deterministic IDs - pas de deux @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate. Eric, Many thanks for engaging in the discourse.   Re: What I would also observe is the sentiment that we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future is flawed. Unless people who do not think they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:  (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the random generator, this is not always the case for Version 4 UUIDs (as described in the RFC). (2) Any random UUIDs generated by Version 4 are easily discernible in the UUID. (3) One can makes inferences from a Version 5 UUID in a Community of Trust where Source Namespaces used to sign and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID. Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee? Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Eric Burger < eric.burger@georgetown.edu > Sent: Wednesday, May 4, 2016 11:50 AM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties To: < cti-stix@lists.oasis-open.org > Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack . My observations: What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used. What I would also observe is the sentiment that we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future is flawed. Unless people who do not think they need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up. Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are getting a UUID generator for free - you won’t have to think about it. On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote: re: So, as with timestamps, there is a difference between I just do not like it and this is the wrong way to go and here is why .    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:  (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.        (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = . 1*DIGIT] The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10 billion bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations. Community members provided real world use cases and challenges where (1) My products operate at these speeds, (2) My use cases require sub-millisecond Timestamps, etc.    Paraphrasing the arguments made against the requested change included (1) My product doesn't operate at these speeds, (2) My use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds, (4) We're not going to pay attention to 'fringe' cases   This is the wrong way to go and here is why :       A small faction should not be able to summarily reject a stated community open requirement on the basis of we don't have or understand this requirement .  re: We have debated this issue almost as long as we have debated timestamps. No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the concept are not accurate.   Paul Patrick has gotten it right.   The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties of the Object   .   So for an Object describing the IP Address 1.2.3.4 , the  Immutable property of the Object = 1.2.3.4 .    The  1.2.3.4   value would NEVER change for this Object.   Any changes to an objects immutable properties would be a new object, not a new version. I'm not going to rehash the rest... Bottom Line: Current language  An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). This is the wrong way to go and here is why :     This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID to Version 4 (random/pseudo random). We get that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a 5 instead of a 4 . Proposed Language An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant UUID. Vote it up or down... Office:   (856)983-0001 Cell:       (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date: Tuesday, May 3, 2016 at 3:09 PM To: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.   We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between I just do not like it and this is the wrong way to go and here is why .  If either the timestamps or IDs that we have proposed are wrong, or are going to make life miserable, please speak up and explain why.  Maybe our resident academic can chime in here?   Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: What would be the proposed third value used in the tuple that would not break versioning? - Jason Keirstead STSM, Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo From: Paul Patrick < ppatrick@isightpartners.com > To: Jordan, Bret < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org > Cc: Wunder, John A. < jwunder@mitre.org >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Date: 05/03/2016 01:37 PM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Sent by: < cti-stix@lists.oasis-open.org > Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email. In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of breaking relationships. From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support. In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless. So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used. I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes. Paul Patrick From: < cti-stix@lists.oasis-open.org > on behalf of Jordan, Bret < bret.jordan@bluecoat.com > Date: Monday, May 2, 2016 at 6:01 PM To: Patrick Maroney < Pmaroney@Specere.org > Cc: Wunder, John A. < jwunder@mitre.org >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Resent-From: < Paul.Patrick@FireEye.com > Pat, Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg. On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote: re: . This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method. Format and language are notional and will need corrections by the editors. Basis: If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation -- They can!!! if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc. -- They can To!!! The method of generation of any UUID can of course be determined from the generated UUID. -- Win-Win for both camps and with no cost that I can discern 4.5.? Identifier Type Name: identifier Status: Review MVP: Yes An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1.?2 Examples { type : indicator , id : indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836 , ... } 4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting. Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue) PublIcNameSpace = NameSpace[.subNameSpace] ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace) (4.5.2.1) Examples NameSpace = ' specere.org ' subNameSpace = 'isac-2' ObjectClass = 'AddressObject' ObjectType = 'ipv4-addr' ObjectValue = '1.2.3.5' AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7 Demonstrate deterministic generation of CTI Object IDs Generates Namespace Prefix deterministic generation of CTI Object IDs Type:AddressObjectType.ipv4-addr Value:1.2.3.4 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30 Type:AddressObjectType.ipv4-addr Value:1.2.3.5 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7 Type:DomainNameType.TLD Value: badguys.com Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac Type:URIObjectType.URL Value: http://www.badguys.com/kickme?Hard Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2 Type:pattern Value: pattern : { type : twigs , properties : [ ], prop1 : { }, prop2 : { }, prop3 : { key : FileObject:hashes/hash/simple_hash_value , operator : equals , value : c38862b4835729d979e7940d72a48172 key : FileObject:file_name , operator : contains , value : abcd.dll key : WinRegistryKeyObject:key , operator : equals , value : .DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7 }, prop4 : { } prop5 : { } key : WinRegistryKeyObject:hive , operator : equals , value : HKEY_USERS key : IPv4AddressObject:hive , operator : equals , value : 1.2.3.4 condition : (prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS } Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: f7634169-2a18-5971-a919-585a4a913b72 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8 4.1.3. Version The version number is in the most significant 4 bits of the time stamp (bits 4 through 7 of the time_hi_and_version field). The following table lists the currently-defined versions for this UUID variant. Msb0 Msb1 Msb2 Msb3 Version Description 0 0 0 1 1 The time-based version specified in this document. 0 0 1 0 2 DCE Security version, with embedded POSIX UIDs. 0 0 1 1 3 The name-based version specified in this document that uses MD5 hashing. 0 1 0 0 4 The randomly or pseudo- randomly generated version specified in this document. 0 1 0 1 5 The name-based version specified in this document that uses SHA-1 hashing. The version is more accurately a sub-type; again, we retain the term for compatibility. Patrick Maroney Office: (856)983-0001 Cell: (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of John Wunder < jwunder@mitre.org > Date: Friday, April 29, 2016 at 4:23 PM To: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties All, As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text. In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process: Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft) Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward. We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic. For this first round, please review the following sections in the STIX 2.0 specification: Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86 This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time. John Attachment: signature.asc Description: Message signed with OpenPGP using GPGMail


  • 8.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-04-2016 18:23




    The reason I say that it doesn’t align with our versioning proposal is that for that approach we specifically said that we wouldn't define that any particular changes to an object would necessarily result in a new object. The determination of what constituted
    a “material change” that would result in a new object was entirely left up to the producer, mainly because of the issue with identifying these “immutable” fields that Bret points out. Any deterministic ID approach that we standardize on would explicitly define
    which changes would result in a new version, taking that “material change” decision out of the hands of the producers and putting it into the hands of us as specification authors.


    John








    From: Patrick Maroney < Pmaroney@specere.org >
    Date: Wednesday, May 4, 2016 at 1:52 PM
    To: Allan Thomson < athomson@lookingglasscyber.com >, "Jordan, Bret" < bret.jordan@bluecoat.com >, Eric Burger < Eric.Burger@georgetown.edu >
    Cc: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, "Wunder, John A." < jwunder@mitre.org >
    Subject: Re: [cti-stix] Deterministic IDs - pas de deux







    Re: "Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change
    and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash."


    Again this is a mis-characterization of my proposal.  The proposed approach is to use the Immutable properties of an Object.  Any changes to these immutable properties are by definition a new object, not a revision. 

    Patrick Maroney
    President
    Integrated Networking Technologies, Inc.
    Desk:
    (856)983-0001
    Cell:
    (609)841-5104
    Email:
    pmaroney@specere.org



    _____________________________
    From: Allan Thomson < athomson@lookingglasscyber.com >
    Sent: Wednesday, May 4, 2016 1:44 PM
    Subject: Re: [cti-stix] Deterministic IDs - pas de deux
    To: Jordan, Bret < bret.jordan@bluecoat.com >, Eric Burger < eric.burger@georgetown.edu >
    Cc: < cti-stix@lists.oasis-open.org >, John A. Wunder < jwunder@mitre.org >




    Why not have a required identifier based on UUID4 (my preference) and an if vendors want a deterministic id then they can include an optional attribute for that. For systems that want to use that optional identifier it would provide support for.


    allan












    From: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com >
    Date: Wednesday, May 4, 2016 at 10:13 AM
    To: Eric Burger < Eric.Burger@georgetown.edu >
    Cc: "Wunder, John" < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
    Subject: Re: [cti-stix] Deterministic IDs - pas de deux






    I agree with Wunder on all of his points. 


    If we decide to do deterministic IDs, than I think we will need to redo versioning, for the reasons that John points out, and others that we do not yet fully realize..  And if we are going to back and redo all of these things, than we should probably
    go back and redo timestamps (there seems to be more people that dislike our timestamps than those that dislike UUIDv4 based IDs).  This would keep parity and fairness in the community.  


    If we do deterministic IDs, then we will need to determine which fields in each TLO will be used, on a TLO by TLO basis.  We will also need to add a field to each TLO that includes namespace and other things that are used for the deterministic
    IDs.











    Thanks,


    Bret











    Bret Jordan CISSP

    Director of Security Architecture and Standards Office of the CTO

    Blue Coat Systems

    PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE

    7415 0050
    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 











    On May 4, 2016, at 11:01, Eric Burger < Eric.Burger@georgetown.edu > wrote:


    I can live with this.
    --
    Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ >
    On May 4, 2016 12:58 PM, "Wunder, John A." < jwunder@mitre.org > wrote:



    I see us having three options:

    We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion.
    Given those choices, in order of preference I would say: 2, 1, 3. My reasoning:

    We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick
    one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing
    UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change
    and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts
    and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach.

    John




    From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org >
    Date: Wednesday, May 4, 2016 at 12:29 PM
    To: Eric Burger < eric.burger@georgetown.edu >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
    Subject: [cti-stix] Deterministic IDs - pas de deux







    @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate.


    Eric,


    Many thanks for engaging in the discourse.  


    Re: "What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think
    they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime."


    I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:


     (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the "random" generator, this is not always the case for Version 4 UUIDs (as described in the RFC).


    (2) Any "random" UUIDs generated by Version 4 are easily discernible in the UUID.


    (3) One can makes inferences from a Version 5 UUID in a "Community of Trust" where Source Namespaces used to "sign" and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID.


    Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee?



    Patrick Maroney
    President
    Integrated Networking Technologies, Inc.
    Desk: (856)983-0001
    Cell: (609)841-5104
    Email: pmaroney@specere.org



    _____________________________
    From: Eric Burger < eric.burger@georgetown.edu >
    Sent: Wednesday, May 4, 2016 11:50 AM
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    To: < cti-stix@lists.oasis-open.org >


    Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack .


    My observations:
    What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used.


    What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need  a
    deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime.


    Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up.


    Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and it simplifies STIX implementations  by have
    one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are getting a UUID generator for free - you won’t have to think about it.



    On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote:




    re: "So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why"."   



    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:



     (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.     
     
    (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = "." 1*DIGIT]






    The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10 billion
    bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations.




    Community members provided real world use cases and challenges where (1) "My" products operate at these speeds, (2) "My" use cases require sub-millisecond Timestamps, etc.    Paraphrasing
    the arguments made against the requested change included (1) "My" product doesn't operate at these speeds, (2) "My" use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds,
    (4) We're not going to pay attention to 'fringe' cases  





    "This is the wrong way to go and here is why":    



      A small faction should not be able to summarily reject a stated community open requirement on the basis of "we don't have or understand this requirement". 









    re: "We have debated this issue almost as long as we have debated timestamps."



    No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the concept
    are not accurate.   Paul Patrick has gotten it right.  


    The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties of
    the Object   .   So for an Object describing the IP Address "1.2.3.4", the  Immutable property of the Object = "1.2.3.4" .    The  "1.2.3.4"  value would NEVER change for this Object.
      Any changes to an objects immutable properties would be a new object, not a new version.


    I'm not going to rehash the rest...


    Bottom Line:



    Current language 










    "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object
    being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID)."






    "This is the wrong way to go and here is why":  
     







    This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID
    to Version 4 (random/pseudo random).


    We "get" that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal
    implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a "5" instead of a "4".






    Proposed Language





    "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object
    being identified or referenced and [uuid] is an RFC 4122 compliant UUID. "





    Vote it up or down...






    Office:   (856)983-0001
    Cell:       (609)841-5104



    <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png>


    President
    Integrated Networking Technologies, Inc.
    PO
    Box 569
    Marlton,
    NJ 08053







    From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com >
    Date: Tuesday, May 3, 2016 at 3:09 PM
    To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu >
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties





    I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.  


    We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why".  If either the timestamps or IDs that we
    have proposed are wrong, or are going to make life miserable, please speak up and explain why. 









    Maybe our resident academic can chime in here?  


    Thanks,


    Bret











    Bret Jordan CISSP

    Director of Security Architecture and Standards Office of the CTO

    Blue Coat Systems

    PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050
    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 











    On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote:



    What would be the proposed third value used in the tuple that would not break versioning?

    -
    Jason Keirstead
    STSM, Product Architect, Security Intelligence, IBM Security Systems
    www.ibm.com/security www.securityintelligence.com

    Without data, all you are is just another person with an opinion - Unknown


    <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo

    From: Paul Patrick < ppatrick@isightpartners.com >
    To: "Jordan, Bret" < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org >
    Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
    Date: 05/03/2016 01:37 PM
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    Sent by: < cti-stix@lists.oasis-open.org >





    Folks,

    I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments
    correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email.

    In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly
    the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the
    hash there isn’t the issue of breaking relationships.

    From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support.
    In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the
    UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless.

    So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used.

    I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes.


    Paul Patrick

    From: < cti-stix@lists.oasis-open.org >
    on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com >
    Date: Monday, May 2, 2016 at 6:01 PM
    To: Patrick Maroney < Pmaroney@Specere.org >
    Cc: "Wunder, John A." < jwunder@mitre.org >,
    " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    Resent-From: < Paul.Patrick@FireEye.com >


    Pat,

    Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need
    to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break.

    Thanks,

    Bret



    Bret Jordan CISSP
    Director of Security Architecture and Standards Office of the CTO
    Blue Coat Systems
    PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050
    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."


    On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote:

    re: . " This is the start of the 2 day review process prior to the initial motion, so please provide
    any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning."


    I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers.
    The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method.

    Format and language are notional and will need corrections by the editors.

    Basis:

    If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation


    -- They can!!!


    if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific
    to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc.



    -- They can To!!!



    The method of generation of any UUID can of course be determined from the generated UUID.



    -- Win-Win for both camps and with no cost that I can discern







    4.5.? Identifier
    Type Name: identifier
    Status: Review
    MVP: Yes

    An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object
    being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

    4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

    4.5.1.?2 Examples
    {
    "type": "indicator",
    "id": "indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836",
    ...
    }

    4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting.

    Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue)

    PublIcNameSpace = NameSpace[.subNameSpace]

    ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace)


    (4.5.2.1) Examples

    NameSpace = ' specere.org '
    subNameSpace = 'isac-2'
    ObjectClass = 'AddressObject'
    ObjectType = 'ipv4-addr'
    ObjectValue = '1.2.3.5'

    AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7

    Demonstrate deterministic generation of CTI Object IDs
    Generates Namespace Prefix deterministic generation of CTI Object IDs

    Type:AddressObjectType.ipv4-addr

    Value:1.2.3.4

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30


    Type:AddressObjectType.ipv4-addr

    Value:1.2.3.5

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7


    Type:DomainNameType.TLD

    Value: badguys.com

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac


    Type:URIObjectType.URL

    Value: http://www.badguys.com/kickme?Hard

    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2


    Type:pattern

    Value:
    "pattern": { "type": "twigs", "properties": [
    ],
    "prop1": {
    }, "prop2": {
    }, "prop3": {
    "key":"FileObject:hashes/hash/simple_hash_value", "operator":"equals", "value":"c38862b4835729d979e7940d72a48172"
    "key":"FileObject:file_name", "operator":"contains", "value":"abcd.dll"
    "key":"WinRegistryKeyObject:key",
    "operator":"equals", "value":".DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7"
    }, "prop4": {
    } "prop5": {
    }
    "key":"WinRegistryKeyObject:hive", "operator":"equals", "value":"HKEY_USERS"
    "key":"IPv4AddressObject:hive", "operator":"equals", "value":"1.2.3.4"
    "condition": "(prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS" }


    Namespace: development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: f7634169-2a18-5971-a919-585a4a913b72

    Namespace: production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56

    Namespace: operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a

    Namespace: isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc

    Namespace: isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8







    4.1.3. Version

    The version number is in the most significant 4 bits of the time
    stamp (bits 4 through 7 of the time_hi_and_version field).

    The following table lists the currently-defined versions for this
    UUID variant.

    Msb0 Msb1 Msb2 Msb3 Version Description

    0 0 0 1 1 The time-based version
    specified in this document.

    0 0 1 0 2 DCE Security version, with
    embedded POSIX UIDs.


    0 0 1 1 3 The name-based version
    specified in this document
    that uses MD5 hashing.

    0 1 0 0 4 The randomly or pseudo-
    randomly generated version
    specified in this document.

    0 1 0 1 5 The name-based version
    specified in this document
    that uses SHA-1 hashing.

    The version is more accurately a sub-type; again, we retain the term
    for compatibility.


    Patrick Maroney
    Office: (856)983-0001
    Cell: (609)841-5104

    <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png>

    President
    Integrated Networking Technologies, Inc.
    PO
    Box 569
    Marlton, NJ 08053

    From: < cti-stix@lists.oasis-open.org >
    on behalf of John Wunder < jwunder@mitre.org >
    Date: Friday, April 29, 2016 at 4:23 PM
    To: " cti-stix@lists.oasis-open.org "
    < cti-stix@lists.oasis-open.org >
    Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties

    All,

    As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development
    cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text.

    In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process:


    Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending
    on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the
    draft status (Status = Draft)

    Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development”
    phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward.

    We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic.

    For this first round, please review the following sections in the STIX 2.0 specification:

    Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q
    Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf
    Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j
    Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86

    This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make
    the motions to approve these sections on Wednesday morning.

    Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time.

    John














































  • 9.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-04-2016 18:27
    And that would mean that any minor type'o changes, that are part of a field used in deterministic IDs would break all relationships.  That seems like a  BAD design to me. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 4, 2016, at 12:22, Wunder, John A. < jwunder@mitre.org > wrote: The reason I say that it doesn’t align with our versioning proposal is that for that approach we specifically said that we wouldn't define that any particular changes to an object would necessarily result in a new object. The determination of what constituted a “material change” that would result in a new object was entirely left up to the producer, mainly because of the issue with identifying these “immutable” fields that Bret points out. Any deterministic ID approach that we standardize on would explicitly define which changes would result in a new version, taking that “material change” decision out of the hands of the producers and putting it into the hands of us as specification authors. John From: Patrick Maroney < Pmaroney@specere.org > Date: Wednesday, May 4, 2016 at 1:52 PM To: Allan Thomson < athomson@lookingglasscyber.com >, Jordan, Bret < bret.jordan@bluecoat.com >, Eric Burger < Eric.Burger@georgetown.edu > Cc: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org >, Wunder, John A. < jwunder@mitre.org > Subject: Re: [cti-stix] Deterministic IDs - pas de deux Re: Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. Again this is a mis-characterization of my proposal.  The proposed approach is to use the Immutable properties of an Object.  Any changes to these immutable properties are by definition a new object, not a revision.  Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Allan Thomson < athomson@lookingglasscyber.com > Sent: Wednesday, May 4, 2016 1:44 PM Subject: Re: [cti-stix] Deterministic IDs - pas de deux To: Jordan, Bret < bret.jordan@bluecoat.com >, Eric Burger < eric.burger@georgetown.edu > Cc: < cti-stix@lists.oasis-open.org >, John A. Wunder < jwunder@mitre.org > Why not have a required identifier based on UUID4 (my preference) and an if vendors want a deterministic id then they can include an optional attribute for that. For systems that want to use that optional identifier it would provide support for. allan From: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > on behalf of Jordan, Bret < bret.jordan@bluecoat.com > Date: Wednesday, May 4, 2016 at 10:13 AM To: Eric Burger < Eric.Burger@georgetown.edu > Cc: Wunder, John < jwunder@mitre.org >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Deterministic IDs - pas de deux I agree with Wunder on all of his points.  If we decide to do deterministic IDs, than I think we will need to redo versioning, for the reasons that John points out, and others that we do not yet fully realize..  And if we are going to back and redo all of these things, than we should probably go back and redo timestamps (there seems to be more people that dislike our timestamps than those that dislike UUIDv4 based IDs).  This would keep parity and fairness in the community.   If we do deterministic IDs, then we will need to determine which fields in each TLO will be used, on a TLO by TLO basis.  We will also need to add a field to each TLO that includes namespace and other things that are used for the deterministic IDs. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 4, 2016, at 11:01, Eric Burger < Eric.Burger@georgetown.edu > wrote: I can live with this. -- Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ > On May 4, 2016 12:58 PM, Wunder, John A. < jwunder@mitre.org > wrote: I see us having three options: We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion. Given those choices, in order of preference I would say: 2, 1, 3. My reasoning: We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach. John From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org > Date: Wednesday, May 4, 2016 at 12:29 PM To: Eric Burger < eric.burger@georgetown.edu >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Deterministic IDs - pas de deux @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate. Eric, Many thanks for engaging in the discourse.   Re: What I would also observe is the sentiment that we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future is flawed. Unless people who do not think they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:  (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the random generator, this is not always the case for Version 4 UUIDs (as described in the RFC). (2) Any random UUIDs generated by Version 4 are easily discernible in the UUID. (3) One can makes inferences from a Version 5 UUID in a Community of Trust where Source Namespaces used to sign and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID. Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee? Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Eric Burger < eric.burger@georgetown.edu > Sent: Wednesday, May 4, 2016 11:50 AM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties To: < cti-stix@lists.oasis-open.org > Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack . My observations: What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used. What I would also observe is the sentiment that we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future is flawed. Unless people who do not think they need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up. Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are getting a UUID generator for free - you won’t have to think about it. On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote: re: So, as with timestamps, there is a difference between I just do not like it and this is the wrong way to go and here is why .    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:  (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.        (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = . 1*DIGIT] The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10 billion bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations. Community members provided real world use cases and challenges where (1) My products operate at these speeds, (2) My use cases require sub-millisecond Timestamps, etc.    Paraphrasing the arguments made against the requested change included (1) My product doesn't operate at these speeds, (2) My use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds, (4) We're not going to pay attention to 'fringe' cases   This is the wrong way to go and here is why :       A small faction should not be able to summarily reject a stated community open requirement on the basis of we don't have or understand this requirement .  re: We have debated this issue almost as long as we have debated timestamps. No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the concept are not accurate.   Paul Patrick has gotten it right.   The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties of the Object   .   So for an Object describing the IP Address 1.2.3.4 , the  Immutable property of the Object = 1.2.3.4 .    The  1.2.3.4   value would NEVER change for this Object.   Any changes to an objects immutable properties would be a new object, not a new version. I'm not going to rehash the rest... Bottom Line: Current language  An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). This is the wrong way to go and here is why :     This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID to Version 4 (random/pseudo random). We get that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a 5 instead of a 4 . Proposed Language An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant UUID. Vote it up or down... Office:   (856)983-0001 Cell:       (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date: Tuesday, May 3, 2016 at 3:09 PM To: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.   We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between I just do not like it and this is the wrong way to go and here is why .  If either the timestamps or IDs that we have proposed are wrong, or are going to make life miserable, please speak up and explain why.  Maybe our resident academic can chime in here?   Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg.   On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: What would be the proposed third value used in the tuple that would not break versioning? - Jason Keirstead STSM, Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo From: Paul Patrick < ppatrick@isightpartners.com > To: Jordan, Bret < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org > Cc: Wunder, John A. < jwunder@mitre.org >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Date: 05/03/2016 01:37 PM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Sent by: < cti-stix@lists.oasis-open.org > Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email. In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of breaking relationships. From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support. In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless. So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used. I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes. Paul Patrick From: < cti-stix@lists.oasis-open.org > on behalf of Jordan, Bret < bret.jordan@bluecoat.com > Date: Monday, May 2, 2016 at 6:01 PM To: Patrick Maroney < Pmaroney@Specere.org > Cc: Wunder, John A. < jwunder@mitre.org >, cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Resent-From: < Paul.Patrick@FireEye.com > Pat, Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg. On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote: re: . This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method. Format and language are notional and will need corrections by the editors. Basis: If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation -- They can!!! if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc. -- They can To!!! The method of generation of any UUID can of course be determined from the generated UUID. -- Win-Win for both camps and with no cost that I can discern 4.5.? Identifier Type Name: identifier Status: Review MVP: Yes An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1.?2 Examples { type : indicator , id : indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836 , ... } 4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting. Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue) PublIcNameSpace = NameSpace[.subNameSpace] ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace) (4.5.2.1) Examples NameSpace = ' specere.org ' subNameSpace = 'isac-2' ObjectClass = 'AddressObject' ObjectType = 'ipv4-addr' ObjectValue = '1.2.3.5' AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7 Demonstrate deterministic generation of CTI Object IDs Generates Namespace Prefix deterministic generation of CTI Object IDs Type:AddressObjectType.ipv4-addr Value:1.2.3.4 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30 Type:AddressObjectType.ipv4-addr Value:1.2.3.5 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7 Type:DomainNameType.TLD Value: badguys.com Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac Type:URIObjectType.URL Value: http://www.badguys.com/kickme?Hard Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2 Type:pattern Value: pattern : { type : twigs , properties : [ ], prop1 : { }, prop2 : { }, prop3 : { key : FileObject:hashes/hash/simple_hash_value , operator : equals , value : c38862b4835729d979e7940d72a48172 key : FileObject:file_name , operator : contains , value : abcd.dll key : WinRegistryKeyObject:key , operator : equals , value : .DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7 }, prop4 : { } prop5 : { } key : WinRegistryKeyObject:hive , operator : equals , value : HKEY_USERS key : IPv4AddressObject:hive , operator : equals , value : 1.2.3.4 condition : (prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS } Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: f7634169-2a18-5971-a919-585a4a913b72 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8 4.1.3. Version The version number is in the most significant 4 bits of the time stamp (bits 4 through 7 of the time_hi_and_version field). The following table lists the currently-defined versions for this UUID variant. Msb0 Msb1 Msb2 Msb3 Version Description 0 0 0 1 1 The time-based version specified in this document. 0 0 1 0 2 DCE Security version, with embedded POSIX UIDs. 0 0 1 1 3 The name-based version specified in this document that uses MD5 hashing. 0 1 0 0 4 The randomly or pseudo- randomly generated version specified in this document. 0 1 0 1 5 The name-based version specified in this document that uses SHA-1 hashing. The version is more accurately a sub-type; again, we retain the term for compatibility. Patrick Maroney Office: (856)983-0001 Cell: (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of John Wunder < jwunder@mitre.org > Date: Friday, April 29, 2016 at 4:23 PM To: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties All, As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text. In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process: Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft) Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward. We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic. For this first round, please review the following sections in the STIX 2.0 specification: Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86 This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time. John Attachment: signature.asc Description: Message signed with OpenPGP using GPGMail


  • 10.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-05-2016 13:45
    As an implementor (aka, programmer), I have wrestled with IDs in the current MITRE Python libraries. After trying many things, my own (drastic!) solution was to monkey-patch the libraries so that I had full control over IDs. I think the core question is this: "Do we compare IDs to determine object equivalence, or must we compare multiple attributes?" It's classic computer science. From: cti-stix@lists.oasis-open.org <cti-stix@lists.oasis-open.org> on behalf of Jordan, Bret <bret.jordan@bluecoat.com> Sent: Wednesday, May 4, 2016 2:27:00 PM To: Wunder, John A. Cc: cti-stix@lists.oasis-open.org Subject: Re: [cti-stix] Deterministic IDs - pas de deux   And that would mean that any minor type'o changes, that are part of a field used in deterministic IDs would break all relationships.  That seems like a  BAD design to me. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 4, 2016, at 12:22, Wunder, John A. < jwunder@mitre.org > wrote: The reason I say that it doesn’t align with our versioning proposal is that for that approach we specifically said that we wouldn't define that any particular changes to an object would necessarily result in a new object. The determination of what constituted a “material change” that would result in a new object was entirely left up to the producer, mainly because of the issue with identifying these “immutable” fields that Bret points out. Any deterministic ID approach that we standardize on would explicitly define which changes would result in a new version, taking that “material change” decision out of the hands of the producers and putting it into the hands of us as specification authors. John From: Patrick Maroney < Pmaroney@specere.org > Date: Wednesday, May 4, 2016 at 1:52 PM To: Allan Thomson < athomson@lookingglasscyber.com >, "Jordan, Bret" < bret.jordan@bluecoat.com >, Eric Burger < Eric.Burger@georgetown.edu > Cc: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, "Wunder, John A." < jwunder@mitre.org > Subject: Re: [cti-stix] Deterministic IDs - pas de deux Re: "Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash." Again this is a mis-characterization of my proposal.  The proposed approach is to use the Immutable properties of an Object.  Any changes to these immutable properties are by definition a new object, not a revision.  Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Allan Thomson < athomson@lookingglasscyber.com > Sent: Wednesday, May 4, 2016 1:44 PM Subject: Re: [cti-stix] Deterministic IDs - pas de deux To: Jordan, Bret < bret.jordan@bluecoat.com >, Eric Burger < eric.burger@georgetown.edu > Cc: < cti-stix@lists.oasis-open.org >, John A. Wunder < jwunder@mitre.org > Why not have a required identifier based on UUID4 (my preference) and an if vendors want a deterministic id then they can include an optional attribute for that. For systems that want to use that optional identifier it would provide support for. allan From: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com > Date: Wednesday, May 4, 2016 at 10:13 AM To: Eric Burger < Eric.Burger@georgetown.edu > Cc: "Wunder, John" < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Deterministic IDs - pas de deux I agree with Wunder on all of his points.  If we decide to do deterministic IDs, than I think we will need to redo versioning, for the reasons that John points out, and others that we do not yet fully realize..  And if we are going to back and redo all of these things, than we should probably go back and redo timestamps (there seems to be more people that dislike our timestamps than those that dislike UUIDv4 based IDs).  This would keep parity and fairness in the community.   If we do deterministic IDs, then we will need to determine which fields in each TLO will be used, on a TLO by TLO basis.  We will also need to add a field to each TLO that includes namespace and other things that are used for the deterministic IDs. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 4, 2016, at 11:01, Eric Burger < Eric.Burger@georgetown.edu > wrote: I can live with this. -- Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ > On May 4, 2016 12:58 PM, "Wunder, John A." < jwunder@mitre.org > wrote: I see us having three options: We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion. Given those choices, in order of preference I would say: 2, 1, 3. My reasoning: We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach. John From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org > Date: Wednesday, May 4, 2016 at 12:29 PM To: Eric Burger < eric.burger@georgetown.edu >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Deterministic IDs - pas de deux @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate. Eric, Many thanks for engaging in the discourse.   Re: "What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime." I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:  (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the "random" generator, this is not always the case for Version 4 UUIDs (as described in the RFC). (2) Any "random" UUIDs generated by Version 4 are easily discernible in the UUID. (3) One can makes inferences from a Version 5 UUID in a "Community of Trust" where Source Namespaces used to "sign" and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID. Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee? Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Eric Burger < eric.burger@georgetown.edu > Sent: Wednesday, May 4, 2016 11:50 AM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties To: < cti-stix@lists.oasis-open.org > Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack . My observations: What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used. What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up. Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are getting a UUID generator for free - you won’t have to think about it. On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote: re: "So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why"."    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:  (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.        (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = "." 1*DIGIT] The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10 billion bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations. Community members provided real world use cases and challenges where (1) "My" products operate at these speeds, (2) "My" use cases require sub-millisecond Timestamps, etc.    Paraphrasing the arguments made against the requested change included (1) "My" product doesn't operate at these speeds, (2) "My" use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds, (4) We're not going to pay attention to 'fringe' cases   "This is the wrong way to go and here is why":       A small faction should not be able to summarily reject a stated community open requirement on the basis of "we don't have or understand this requirement".  re: "We have debated this issue almost as long as we have debated timestamps." No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the concept are not accurate.   Paul Patrick has gotten it right.   The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties of the Object   .   So for an Object describing the IP Address "1.2.3.4", the  Immutable property of the Object = "1.2.3.4" .    The  "1.2.3.4"  value would NEVER change for this Object.   Any changes to an objects immutable properties would be a new object, not a new version. I'm not going to rehash the rest... Bottom Line: Current language  "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID)." "This is the wrong way to go and here is why":     This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID to Version 4 (random/pseudo random). We "get" that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a "5" instead of a "4". Proposed Language "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant UUID. " Vote it up or down... Office:   (856)983-0001 Cell:       (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date: Tuesday, May 3, 2016 at 3:09 PM To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.   We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why".  If either the timestamps or IDs that we have proposed are wrong, or are going to make life miserable, please speak up and explain why.  Maybe our resident academic can chime in here?   Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: What would be the proposed third value used in the tuple that would not break versioning? - Jason Keirstead STSM, Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo From: Paul Patrick < ppatrick@isightpartners.com > To: "Jordan, Bret" < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org > Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Date: 05/03/2016 01:37 PM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Sent by: < cti-stix@lists.oasis-open.org > Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email. In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of breaking relationships. From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support. In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless. So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used. I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes. Paul Patrick From: < cti-stix@lists.oasis-open.org > on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com > Date: Monday, May 2, 2016 at 6:01 PM To: Patrick Maroney < Pmaroney@Specere.org > Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Resent-From: < Paul.Patrick@FireEye.com > Pat, Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote: re: . " This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning." I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method. Format and language are notional and will need corrections by the editors. Basis: If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation -- They can!!! if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc. -- They can To!!! The method of generation of any UUID can of course be determined from the generated UUID. -- Win-Win for both camps and with no cost that I can discern 4.5.? Identifier Type Name: identifier Status: Review MVP: Yes An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1.?2 Examples { "type": "indicator", "id": "indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836", ... } 4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting. Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue) PublIcNameSpace = NameSpace[.subNameSpace] ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace) (4.5.2.1) Examples NameSpace = ' specere.org ' subNameSpace = 'isac-2' ObjectClass = 'AddressObject' ObjectType = 'ipv4-addr' ObjectValue = '1.2.3.5' AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7 Demonstrate deterministic generation of CTI Object IDs Generates Namespace Prefix deterministic generation of CTI Object IDs Type:AddressObjectType.ipv4-addr Value:1.2.3.4 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30 Type:AddressObjectType.ipv4-addr Value:1.2.3.5 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7 Type:DomainNameType.TLD Value: badguys.com Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac Type:URIObjectType.URL Value: http://www.badguys.com/kickme?Hard Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2 Type:pattern Value: "pattern": { "type": "twigs", "properties": [ ], "prop1": { }, "prop2": { }, "prop3": { "key":"FileObject:hashes/hash/simple_hash_value", "operator":"equals", "value":"c38862b4835729d979e7940d72a48172" "key":"FileObject:file_name", "operator":"contains", "value":"abcd.dll" "key":"WinRegistryKeyObject:key", "operator":"equals", "value":".DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7" }, "prop4": { } "prop5": { } "key":"WinRegistryKeyObject:hive", "operator":"equals", "value":"HKEY_USERS" "key":"IPv4AddressObject:hive", "operator":"equals", "value":"1.2.3.4" "condition": "(prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS" } Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: f7634169-2a18-5971-a919-585a4a913b72 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8 4.1.3. Version The version number is in the most significant 4 bits of the time stamp (bits 4 through 7 of the time_hi_and_version field). The following table lists the currently-defined versions for this UUID variant. Msb0 Msb1 Msb2 Msb3 Version Description 0 0 0 1 1 The time-based version specified in this document. 0 0 1 0 2 DCE Security version, with embedded POSIX UIDs. 0 0 1 1 3 The name-based version specified in this document that uses MD5 hashing. 0 1 0 0 4 The randomly or pseudo- randomly generated version specified in this document. 0 1 0 1 5 The name-based version specified in this document that uses SHA-1 hashing. The version is more accurately a sub-type; again, we retain the term for compatibility. Patrick Maroney Office: (856)983-0001 Cell: (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of John Wunder < jwunder@mitre.org > Date: Friday, April 29, 2016 at 4:23 PM To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties All, As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text. In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process: Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft) Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward. We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic. For this first round, please review the following sections in the STIX 2.0 specification: Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86 This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time. John


  • 11.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-05-2016 19:12
    One closing comment on this thread (as the original thread creator): The proposed Deterministic ID concept has been mis-characterized, and rejected on this basis from the beginning.  However, the discourse (before yesterday's attempt to shut down active discussions amongst other voices in the community) has surfaced a much bigger issue from my perspective. On the one hand the initial attempts to reach a reasonable compromise on Identity specification by simply allowing the object creator to choose the method of RFC 4122 compliant UUID was rejected on the basis of "one way of doing things".  But these same voices then go on state that something as critical as the criteria for what constitutes the difference between a new object and a revision to an existing object is totally subjective and left up to each individual object creator. Really? Fait Accompli - consider me disengaged. Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: John Anderson < janderson@soltra.com > Sent: Thursday, May 5, 2016 9:45 AM Subject: Re: [cti-stix] Deterministic IDs - pas de deux To: Jordan, Bret < bret.jordan@bluecoat.com >, Wunder, John A. < jwunder@mitre.org > Cc: < cti-stix@lists.oasis-open.org > As an implementor (aka, programmer), I have wrestled with IDs in the current MITRE Python libraries. After trying many things, my own (drastic!) solution was to monkey-patch the libraries so that I had full control over IDs. I think the core question is this: "Do we compare IDs to determine object equivalence, or must we compare multiple attributes?" It's classic computer science. From: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > on behalf of Jordan, Bret < bret.jordan@bluecoat.com > Sent: Wednesday, May 4, 2016 2:27:00 PM To: Wunder, John A. Cc: cti-stix@lists.oasis-open.org Subject: Re: [cti-stix] Deterministic IDs - pas de deux   And that would mean that any minor type'o changes, that are part of a field used in deterministic IDs would break all relationships.  That seems like a  BAD design to me. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 4, 2016, at 12:22, Wunder, John A. < jwunder@mitre.org > wrote: The reason I say that it doesn’t align with our versioning proposal is that for that approach we specifically said that we wouldn't define that any particular changes to an object would necessarily result in a new object. The determination of what constituted a “material change” that would result in a new object was entirely left up to the producer, mainly because of the issue with identifying these “immutable” fields that Bret points out. Any deterministic ID approach that we standardize on would explicitly define which changes would result in a new version, taking that “material change” decision out of the hands of the producers and putting it into the hands of us as specification authors. John From: Patrick Maroney < Pmaroney@specere.org > Date: Wednesday, May 4, 2016 at 1:52 PM To: Allan Thomson < athomson@lookingglasscyber.com >, "Jordan, Bret" < bret.jordan@bluecoat.com >, Eric Burger < Eric.Burger@georgetown.edu > Cc: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, "Wunder, John A." < jwunder@mitre.org > Subject: Re: [cti-stix] Deterministic IDs - pas de deux Re: "Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash." Again this is a mis-characterization of my proposal.  The proposed approach is to use the Immutable properties of an Object.  Any changes to these immutable properties are by definition a new object, not a revision.  Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Allan Thomson < athomson@lookingglasscyber.com > Sent: Wednesday, May 4, 2016 1:44 PM Subject: Re: [cti-stix] Deterministic IDs - pas de deux To: Jordan, Bret < bret.jordan@bluecoat.com >, Eric Burger < eric.burger@georgetown.edu > Cc: < cti-stix@lists.oasis-open.org >, John A. Wunder < jwunder@mitre.org > Why not have a required identifier based on UUID4 (my preference) and an if vendors want a deterministic id then they can include an optional attribute for that. For systems that want to use that optional identifier it would provide support for. allan From: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com > Date: Wednesday, May 4, 2016 at 10:13 AM To: Eric Burger < Eric.Burger@georgetown.edu > Cc: "Wunder, John" < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Deterministic IDs - pas de deux I agree with Wunder on all of his points.  If we decide to do deterministic IDs, than I think we will need to redo versioning, for the reasons that John points out, and others that we do not yet fully realize..  And if we are going to back and redo all of these things, than we should probably go back and redo timestamps (there seems to be more people that dislike our timestamps than those that dislike UUIDv4 based IDs).  This would keep parity and fairness in the community.   If we do deterministic IDs, then we will need to determine which fields in each TLO will be used, on a TLO by TLO basis.  We will also need to add a field to each TLO that includes namespace and other things that are used for the deterministic IDs. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 4, 2016, at 11:01, Eric Burger < Eric.Burger@georgetown.edu > wrote: I can live with this. -- Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ > On May 4, 2016 12:58 PM, "Wunder, John A." < jwunder@mitre.org > wrote: I see us having three options: We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion. Given those choices, in order of preference I would say: 2, 1, 3. My reasoning: We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach. John From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org > Date: Wednesday, May 4, 2016 at 12:29 PM To: Eric Burger < eric.burger@georgetown.edu >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Deterministic IDs - pas de deux @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate. Eric, Many thanks for engaging in the discourse.   Re: "What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime." I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:  (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the "random" generator, this is not always the case for Version 4 UUIDs (as described in the RFC). (2) Any "random" UUIDs generated by Version 4 are easily discernible in the UUID. (3) One can makes inferences from a Version 5 UUID in a "Community of Trust" where Source Namespaces used to "sign" and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID. Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee? Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Eric Burger < eric.burger@georgetown.edu > Sent: Wednesday, May 4, 2016 11:50 AM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties To: < cti-stix@lists.oasis-open.org > Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack . My observations: What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used. What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up. Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are getting a UUID generator for free - you won’t have to think about it. On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote: re: "So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why"."    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:  (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.        (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = "." 1*DIGIT] The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10 billion bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations. Community members provided real world use cases and challenges where (1) "My" products operate at these speeds, (2) "My" use cases require sub-millisecond Timestamps, etc.    Paraphrasing the arguments made against the requested change included (1) "My" product doesn't operate at these speeds, (2) "My" use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds, (4) We're not going to pay attention to 'fringe' cases   "This is the wrong way to go and here is why":       A small faction should not be able to summarily reject a stated community open requirement on the basis of "we don't have or understand this requirement".  re: "We have debated this issue almost as long as we have debated timestamps." No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the concept are not accurate.   Paul Patrick has gotten it right.   The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties of the Object   .   So for an Object describing the IP Address "1.2.3.4", the  Immutable property of the Object = "1.2.3.4" .    The  "1.2.3.4"  value would NEVER change for this Object.   Any changes to an objects immutable properties would be a new object, not a new version. I'm not going to rehash the rest... Bottom Line: Current language  "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID)." "This is the wrong way to go and here is why":     This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID to Version 4 (random/pseudo random). We "get" that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a "5" instead of a "4". Proposed Language "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant UUID. " Vote it up or down... Office:   (856)983-0001 Cell:       (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date: Tuesday, May 3, 2016 at 3:09 PM To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.   We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why".  If either the timestamps or IDs that we have proposed are wrong, or are going to make life miserable, please speak up and explain why.  Maybe our resident academic can chime in here?   Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: What would be the proposed third value used in the tuple that would not break versioning? - Jason Keirstead STSM, Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo From: Paul Patrick < ppatrick@isightpartners.com > To: "Jordan, Bret" < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org > Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Date: 05/03/2016 01:37 PM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Sent by: < cti-stix@lists.oasis-open.org > Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email. In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of breaking relationships. From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support. In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless. So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used. I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes. Paul Patrick From: < cti-stix@lists.oasis-open.org > on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com > Date: Monday, May 2, 2016 at 6:01 PM To: Patrick Maroney < Pmaroney@Specere.org > Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Resent-From: < Paul.Patrick@FireEye.com > Pat, Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote: re: . " This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning." I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method. Format and language are notional and will need corrections by the editors. Basis: If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation -- They can!!! if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc. -- They can To!!! The method of generation of any UUID can of course be determined from the generated UUID. -- Win-Win for both camps and with no cost that I can discern 4.5.? Identifier Type Name: identifier Status: Review MVP: Yes An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1.?2 Examples { "type": "indicator", "id": "indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836", ... } 4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting. Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue) PublIcNameSpace = NameSpace[.subNameSpace] ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace) (4.5.2.1) Examples NameSpace = ' specere.org ' subNameSpace = 'isac-2' ObjectClass = 'AddressObject' ObjectType = 'ipv4-addr' ObjectValue = '1.2.3.5' AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7 Demonstrate deterministic generation of CTI Object IDs Generates Namespace Prefix deterministic generation of CTI Object IDs Type:AddressObjectType.ipv4-addr Value:1.2.3.4 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30 Type:AddressObjectType.ipv4-addr Value:1.2.3.5 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7 Type:DomainNameType.TLD Value: badguys.com Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac Type:URIObjectType.URL Value: http://www.badguys.com/kickme?Hard Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2 Type:pattern Value: "pattern": { "type": "twigs", "properties": [ ], "prop1": { }, "prop2": { }, "prop3": { "key":"FileObject:hashes/hash/simple_hash_value", "operator":"equals", "value":"c38862b4835729d979e7940d72a48172" "key":"FileObject:file_name", "operator":"contains", "value":"abcd.dll" "key":"WinRegistryKeyObject:key", "operator":"equals", "value":".DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7" }, "prop4": { } "prop5": { } "key":"WinRegistryKeyObject:hive", "operator":"equals", "value":"HKEY_USERS" "key":"IPv4AddressObject:hive", "operator":"equals", "value":"1.2.3.4" "condition": "(prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS" } Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: f7634169-2a18-5971-a919-585a4a913b72 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8 4.1.3. Version The version number is in the most significant 4 bits of the time stamp (bits 4 through 7 of the time_hi_and_version field). The following table lists the currently-defined versions for this UUID variant. Msb0 Msb1 Msb2 Msb3 Version Description 0 0 0 1 1 The time-based version specified in this document. 0 0 1 0 2 DCE Security version, with embedded POSIX UIDs. 0 0 1 1 3 The name-based version specified in this document that uses MD5 hashing. 0 1 0 0 4 The randomly or pseudo- randomly generated version specified in this document. 0 1 0 1 5 The name-based version specified in this document that uses SHA-1 hashing. The version is more accurately a sub-type; again, we retain the term for compatibility. Patrick Maroney Office: (856)983-0001 Cell: (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of John Wunder < jwunder@mitre.org > Date: Friday, April 29, 2016 at 4:23 PM To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties All, As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text. In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process: Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft) Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward. We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic. For this first round, please review the following sections in the STIX 2.0 specification: Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86 This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time. John


  • 12.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-05-2016 20:09
    Patrick (if you're reading this thread still) and everyone, I'm not sure if my 2 cents was the straw that broke the camel's back or not. If so, my sincere apologies. Completely not my intent. I meant to communicate two things: (a) I'm not really sure what the answer is and (b) I'm interested in hearing the final answer, whenever we (read: smarter people than me) figure it out. Sincerely, John Anderson From: Patrick Maroney <Pmaroney@Specere.org> Sent: Thursday, May 5, 2016 3:11:37 PM To: John Anderson; Jordan, Bret; Wunder, John A. Cc: cti-stix@lists.oasis-open.org Subject: Re: [cti-stix] Deterministic IDs - pas de deux   One closing comment on this thread (as the original thread creator): The proposed Deterministic ID concept has been mis-characterized, and rejected on this basis from the beginning.  However, the discourse (before yesterday's attempt to shut down active discussions amongst other voices in the community) has surfaced a much bigger issue from my perspective. On the one hand the initial attempts to reach a reasonable compromise on Identity specification by simply allowing the object creator to choose the method of RFC 4122 compliant UUID was rejected on the basis of "one way of doing things".  But these same voices then go on state that something as critical as the criteria for what constitutes the difference between a new object and a revision to an existing object is totally subjective and left up to each individual object creator. Really? Fait Accompli - consider me disengaged. Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: John Anderson < janderson@soltra.com > Sent: Thursday, May 5, 2016 9:45 AM Subject: Re: [cti-stix] Deterministic IDs - pas de deux To: Jordan, Bret < bret.jordan@bluecoat.com >, Wunder, John A. < jwunder@mitre.org > Cc: < cti-stix@lists.oasis-open.org > As an implementor (aka, programmer), I have wrestled with IDs in the current MITRE Python libraries. After trying many things, my own (drastic!) solution was to monkey-patch the libraries so that I had full control over IDs. I think the core question is this: "Do we compare IDs to determine object equivalence, or must we compare multiple attributes?" It's classic computer science. From: cti-stix@lists.oasis-open.org < cti-stix@lists.oasis-open.org > on behalf of Jordan, Bret < bret.jordan@bluecoat.com > Sent: Wednesday, May 4, 2016 2:27:00 PM To: Wunder, John A. Cc: cti-stix@lists.oasis-open.org Subject: Re: [cti-stix] Deterministic IDs - pas de deux   And that would mean that any minor type'o changes, that are part of a field used in deterministic IDs would break all relationships.  That seems like a  BAD design to me. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 4, 2016, at 12:22, Wunder, John A. < jwunder@mitre.org > wrote: The reason I say that it doesn’t align with our versioning proposal is that for that approach we specifically said that we wouldn't define that any particular changes to an object would necessarily result in a new object. The determination of what constituted a “material change” that would result in a new object was entirely left up to the producer, mainly because of the issue with identifying these “immutable” fields that Bret points out. Any deterministic ID approach that we standardize on would explicitly define which changes would result in a new version, taking that “material change” decision out of the hands of the producers and putting it into the hands of us as specification authors. John From: Patrick Maroney < Pmaroney@specere.org > Date: Wednesday, May 4, 2016 at 1:52 PM To: Allan Thomson < athomson@lookingglasscyber.com >, "Jordan, Bret" < bret.jordan@bluecoat.com >, Eric Burger < Eric.Burger@georgetown.edu > Cc: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, "Wunder, John A." < jwunder@mitre.org > Subject: Re: [cti-stix] Deterministic IDs - pas de deux Re: "Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash." Again this is a mis-characterization of my proposal.  The proposed approach is to use the Immutable properties of an Object.  Any changes to these immutable properties are by definition a new object, not a revision.  Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Allan Thomson < athomson@lookingglasscyber.com > Sent: Wednesday, May 4, 2016 1:44 PM Subject: Re: [cti-stix] Deterministic IDs - pas de deux To: Jordan, Bret < bret.jordan@bluecoat.com >, Eric Burger < eric.burger@georgetown.edu > Cc: < cti-stix@lists.oasis-open.org >, John A. Wunder < jwunder@mitre.org > Why not have a required identifier based on UUID4 (my preference) and an if vendors want a deterministic id then they can include an optional attribute for that. For systems that want to use that optional identifier it would provide support for. allan From: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com > Date: Wednesday, May 4, 2016 at 10:13 AM To: Eric Burger < Eric.Burger@georgetown.edu > Cc: "Wunder, John" < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Deterministic IDs - pas de deux I agree with Wunder on all of his points.  If we decide to do deterministic IDs, than I think we will need to redo versioning, for the reasons that John points out, and others that we do not yet fully realize..  And if we are going to back and redo all of these things, than we should probably go back and redo timestamps (there seems to be more people that dislike our timestamps than those that dislike UUIDv4 based IDs).  This would keep parity and fairness in the community.   If we do deterministic IDs, then we will need to determine which fields in each TLO will be used, on a TLO by TLO basis.  We will also need to add a field to each TLO that includes namespace and other things that are used for the deterministic IDs. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 4, 2016, at 11:01, Eric Burger < Eric.Burger@georgetown.edu > wrote: I can live with this. -- Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ > On May 4, 2016 12:58 PM, "Wunder, John A." < jwunder@mitre.org > wrote: I see us having three options: We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting. We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what the current text states. We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion. Given those choices, in order of preference I would say: 2, 1, 3. My reasoning: We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices and usages of the ID field that will cause incompatibilities down the line. We should pick one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having some people doing it one way, other people doing it a different way, and most people doing UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required. Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields included in your hash. The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach. John From: < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org > Date: Wednesday, May 4, 2016 at 12:29 PM To: Eric Burger < eric.burger@georgetown.edu >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Deterministic IDs - pas de deux @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested in further exploring this topic.  I'll rename the subject so those not interested can discriminate. Eric, Many thanks for engaging in the discourse.   Re: "What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime." I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can see are:  (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the "random" generator, this is not always the case for Version 4 UUIDs (as described in the RFC). (2) Any "random" UUIDs generated by Version 4 are easily discernible in the UUID. (3) One can makes inferences from a Version 5 UUID in a "Community of Trust" where Source Namespaces used to "sign" and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID. Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee? Patrick Maroney President Integrated Networking Technologies, Inc. Desk: (856)983-0001 Cell: (609)841-5104 Email: pmaroney@specere.org _____________________________ From: Eric Burger < eric.burger@georgetown.edu > Sent: Wednesday, May 4, 2016 11:50 AM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties To: < cti-stix@lists.oasis-open.org > Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack . My observations: What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful because it is being used, and (3) is easy to implement, because it is being used. What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime. Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message there will be the opportunity to screw it up. Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are getting a UUID generator for free - you won’t have to think about it. On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote: re: "So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why"."    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:  (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are observing, measuring, and conveying to others.        (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = "." 1*DIGIT] The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in November 2013:  For example 10GbE (10×10^9 or 10 billion bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations. Community members provided real world use cases and challenges where (1) "My" products operate at these speeds, (2) "My" use cases require sub-millisecond Timestamps, etc.    Paraphrasing the arguments made against the requested change included (1) "My" product doesn't operate at these speeds, (2) "My" use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application, so I don't see the need for fractional seconds, (4) We're not going to pay attention to 'fringe' cases   "This is the wrong way to go and here is why":       A small faction should not be able to summarily reject a stated community open requirement on the basis of "we don't have or understand this requirement".  re: "We have debated this issue almost as long as we have debated timestamps." No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are making to the list as the basis for rejecting the concept are not accurate.   Paul Patrick has gotten it right.   The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and (3)  Immutable properties of the Object   .   So for an Object describing the IP Address "1.2.3.4", the  Immutable property of the Object = "1.2.3.4" .    The  "1.2.3.4"  value would NEVER change for this Object.   Any changes to an objects immutable properties would be a new object, not a new version. I'm not going to rehash the rest... Bottom Line: Current language  "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID)." "This is the wrong way to go and here is why":     This language unnecessarily and arbitrarily constrains the method for the generation of the  RFC 4122 variant  UUID to Version 4 (random/pseudo random). We "get" that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of us who have valid Use Cases for Deterministic IDs as part of our internal implementation details.  They will be indistinguishable from any other UUID except for the  most significant 4 bits of the time stamp will be a "5" instead of a "4". Proposed Language "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant UUID. " Vote it up or down... Office:   (856)983-0001 Cell:       (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com > Date: Tuesday, May 3, 2016 at 3:09 PM To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.   We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why".  If either the timestamps or IDs that we have proposed are wrong, or are going to make life miserable, please speak up and explain why.  Maybe our resident academic can chime in here?   Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."  On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: What would be the proposed third value used in the tuple that would not break versioning? - Jason Keirstead STSM, Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo From: Paul Patrick < ppatrick@isightpartners.com > To: "Jordan, Bret" < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org > Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Date: 05/03/2016 01:37 PM Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Sent by: < cti-stix@lists.oasis-open.org > Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email. In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of breaking relationships. From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support. In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless. So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used. I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes. Paul Patrick From: < cti-stix@lists.oasis-open.org > on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com > Date: Monday, May 2, 2016 at 6:01 PM To: Patrick Maroney < Pmaroney@Specere.org > Cc: "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties Resent-From: < Paul.Patrick@FireEye.com > Pat, Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote: re: . " This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning." I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method. Format and language are notional and will need corrections by the editors. Basis: If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation -- They can!!! if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc. -- They can To!!! The method of generation of any UUID can of course be determined from the generated UUID. -- Win-Win for both camps and with no cost that I can discern 4.5.? Identifier Type Name: identifier Status: Review MVP: Yes An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID). 4.5.1.?2 Examples { "type": "indicator", "id": "indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836", ... } 4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting. Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue) PublIcNameSpace = NameSpace[.subNameSpace] ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace) (4.5.2.1) Examples NameSpace = ' specere.org ' subNameSpace = 'isac-2' ObjectClass = 'AddressObject' ObjectType = 'ipv4-addr' ObjectValue = '1.2.3.5' AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7 Demonstrate deterministic generation of CTI Object IDs Generates Namespace Prefix deterministic generation of CTI Object IDs Type:AddressObjectType.ipv4-addr Value:1.2.3.4 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30 Type:AddressObjectType.ipv4-addr Value:1.2.3.5 Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7 Type:DomainNameType.TLD Value: badguys.com Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3 Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac Type:URIObjectType.URL Value: http://www.badguys.com/kickme?Hard Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03 Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2 Type:pattern Value: "pattern": { "type": "twigs", "properties": [ ], "prop1": { }, "prop2": { }, "prop3": { "key":"FileObject:hashes/hash/simple_hash_value", "operator":"equals", "value":"c38862b4835729d979e7940d72a48172" "key":"FileObject:file_name", "operator":"contains", "value":"abcd.dll" "key":"WinRegistryKeyObject:key", "operator":"equals", "value":".DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7" }, "prop4": { } "prop5": { } "key":"WinRegistryKeyObject:hive", "operator":"equals", "value":"HKEY_USERS" "key":"IPv4AddressObject:hive", "operator":"equals", "value":"1.2.3.4" "condition": "(prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS" } Namespace: development.specere.org Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a ID Suffix: f7634169-2a18-5971-a919-585a4a913b72 Namespace: production.specere.org Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56 Namespace: operational-testing.specere.org Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765 ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a Namespace: isac-1.specere.org Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910 ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc Namespace: isac-2.specere.org Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8 4.1.3. Version The version number is in the most significant 4 bits of the time stamp (bits 4 through 7 of the time_hi_and_version field). The following table lists the currently-defined versions for this UUID variant. Msb0 Msb1 Msb2 Msb3 Version Description 0 0 0 1 1 The time-based version specified in this document. 0 0 1 0 2 DCE Security version, with embedded POSIX UIDs. 0 0 1 1 3 The name-based version specified in this document that uses MD5 hashing. 0 1 0 0 4 The randomly or pseudo- randomly generated version specified in this document. 0 1 0 1 5 The name-based version specified in this document that uses SHA-1 hashing. The version is more accurately a sub-type; again, we retain the term for compatibility. Patrick Maroney Office: (856)983-0001 Cell: (609)841-5104 <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png> President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: < cti-stix@lists.oasis-open.org > on behalf of John Wunder < jwunder@mitre.org > Date: Friday, April 29, 2016 at 4:23 PM To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties All, As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text. In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process: Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development) Normative text is developed, iterated, and settles down (Status = Development) We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review) After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft) Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward. We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic. For this first round, please review the following sections in the STIX 2.0 specification: Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86 This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning. Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time. John


  • 13.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-06-2016 09:09
    On 05.05.2016 19:11:37, Patrick Maroney wrote: > > The proposed Deterministic ID concept has been mis-characterized, > and rejected on this basis from the beginning. However, the > discourse (before yesterday's attempt to shut down active > discussions amongst other voices in the community) has surfaced a > much bigger issue from my perspective. > Hey, Pat - I'm sorry to hear that you feel your input has been suppressed. From where I sit, there's been a long and healthy discussion around the concept of deterministic identifiers. Personally, I wish we could do them for a handful of interesting applications, but after going back and forth at great length, it just doesn't appear to be practical. The TC is a democratic body. Sometimes democratic bodies take decisions we disagree with as individuals. Take, for example, the CTI Common ballot measure I put forward. I was flabbergasted at the level of opposition to something that seemed so obvious to me, but I didn't take it personally. That's just how democracy works. > > Fait Accompli - consider me disengaged. > Please don't go, Pat. The TC as a whole would be measurably impoverished by the loss of your input. -- Cheers, Trey -- Trey Darley Senior Security Engineer 4DAA 0A88 34BC 27C9 FD2B A97E D3C6 5C74 0FB7 E430 Soltra An FS-ISAC & DTCC Company www.soltra.com -- "It is easier to move a problem around (for example, by moving the problem to a different part of the overall network architecture) than it is to solve it." --RFC 1925 Attachment: signature.asc Description: Digital signature


  • 14.  RE: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-04-2016 18:11




    I think this might be a reasonable compromise – because Pat states: 

     
    (3) One can makes inferences from a Version 5 UUID in a "Community of Trust" where Source Namespaces used to "sign" and objects are shared.
      Otherwise one can infer nothing else from a Version 4 or Version 5 UUID.
     
    If you are in a community of trust you would know that you should use the optional property for whatever purpose your community has agreed to.  Others outside
    that community would ignore that property. 
     
    Personally, I think the same could be true if UUID 5s could be used in the id field itself, but since people seem to think that might cause problems – this could
    work for everyone.
     
    One more thing, Pat – does every TLO have immutable properties?   As John mentioned, you are referring to CybOX objects, so it might be true for them, but for
    STIX objects, may not.
    CybOX btw, could do something different as far as ids go…
     
     


    From: cti-stix@lists.oasis-open.org [mailto:cti-stix@lists.oasis-open.org]
    On Behalf Of Allan Thomson
    Sent: Wednesday, May 04, 2016 1:44 PM
    To: Jordan, Bret <bret.jordan@bluecoat.com>; Eric Burger <Eric.Burger@georgetown.edu>
    Cc: Wunder, John A. <jwunder@mitre.org>; cti-stix@lists.oasis-open.org
    Subject: Re: [cti-stix] Deterministic IDs - pas de deux


     


    Why not have a required identifier based on UUID4 (my preference) and an if vendors want a deterministic id then they can include an optional
    attribute for that. For systems that want to use that optional identifier it would provide support for.


     


    allan


     


     



     


    From:
    " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > on behalf of "Jordan,
    Bret" < bret.jordan@bluecoat.com >
    Date: Wednesday, May 4, 2016 at 10:13 AM
    To: Eric Burger < Eric.Burger@georgetown.edu >
    Cc: "Wunder, John" < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
    Subject: Re: [cti-stix] Deterministic IDs - pas de deux


     



    I agree with Wunder on all of his points. 


     


    If we decide to do deterministic IDs, than I think we will need to redo versioning, for the reasons that John points out, and others that
    we do not yet fully realize..  And if we are going to back and redo all of these things, than we should probably go back and redo timestamps (there seems to be more people that dislike our timestamps than those that dislike UUIDv4 based IDs).  This would keep
    parity and fairness in the community.  


     


    If we do deterministic IDs, then we will need to determine which fields in each TLO will be used, on a TLO by TLO basis.  We will also
    need to add a field to each TLO that includes namespace and other things that are used for the deterministic IDs.








     


    Thanks,


     


    Bret



     


     


     



    Bret Jordan CISSP


    Director of Security Architecture and Standards Office of the CTO


    Blue Coat Systems



    PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050


    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 









     



    On May 4, 2016, at 11:01, Eric Burger < Eric.Burger@georgetown.edu > wrote:

     


    I can live with this.


    --
    Sent from a mobile device. Sorry for typos or weird auto-correct. Thank IETF LEMONADE for mobile email! See < http://www.standardstrack.com/ietf/lemonade/ >

    On May 4, 2016 12:58 PM, "Wunder, John A." < jwunder@mitre.org > wrote:




    I see us having three options:


    1.       
    We define an approach to doing deterministic IDs in the standard, and require that people use it. I believe this is what Eric is suggesting.

    2.       
    We define that we will not do deterministic IDs, use UUID4, and those wanting content hashes/correlation IDs can do it in a custom field. This is what
    the current text states.

    3.       
    We leave it open to implementors to decide whether they do deterministic or non-deterministic IDs. This is Pat’s suggestion.

    Given those choices, in order of preference I would say: 2, 1, 3. My reasoning:


    ·         
    We’re a standards body and should whenever possible identify standard ways of doing things. Leaving things like this open will lead to divergent practices
    and usages of the ID field that will cause incompatibilities down the line. We should pick one way and do it, and let those who want to do custom things use custom fields to do so. So, if we do deterministic IDs, we should go all out and actually do it. Having
    some people doing it one way, other people doing it a different way, and most people doing UUID4 is IMO not a good argument for the standard. We lose out on a lot of the value of deterministic IDs if we make it optional instead of required.

    ·         
    Deterministic IDs will break the proposed versioning approach, which has had very good agreement both on slack and at the F2F. I say this because versioning
    requires that we give producers the ability to determine what constitutes a material change and, when they determine that, let them create a new ID for the construct. Using a deterministic ID precludes this…ID changes would be defined by changes to the fields
    included in your hash.

    ·         
    The use case for deterministic IDs is definitely iffier for me than in CybOX (I.e., Pat’s IP address example is for CybOX, not STIX). What would it look
    like for indicator, observation, sighting, actor, campaign, etc.? Those are evolving concepts and defining a set of fields that you use in every case to match exactly and do correlation might not be the right approach.


    John


     


    From:
    < cti-stix@lists.oasis-open.org > on behalf of Patrick Maroney < Pmaroney@Specere.org >
    Date: Wednesday, May 4, 2016 at 12:29 PM
    To: Eric Burger < eric.burger@georgetown.edu >, " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
    Subject: [cti-stix] Deterministic IDs - pas de deux


     





    @All: I retract my assertion that there has been no discourse on this and look forward to engaging with any in the community interested
    in further exploring this topic.  I'll rename the subject so those not interested can discriminate.


     


    Eric,


     


    Many thanks for engaging in the discourse.  


     


    Re: "What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually
    discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they need a deterministic UUID generator happen to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new,
    post-deterministic UUID regime."


     


    I'd like to better understand where you see potential issues.  .  From a practical DB perspective the only operational differences I can
    see are:


     


     (1) Deterministic IDs are guaranteed to have no collisions. Depending on the implementation of the "random" generator, this is not always
    the case for Version 4 UUIDs (as described in the RFC).


     


    (2) Any "random" UUIDs generated by Version 4 are easily discernible in the UUID.


     


    (3) One can makes inferences from a Version 5 UUID in a "Community of Trust" where Source Namespaces used to "sign" and objects are shared.
      Otherwise one can infer nothing else from a Version 4 or Version 5 UUID.


     


    Given the primary structural reasons for Object IDs (as unique references) what potential STIX incompatibilities do you foresee?


     



    Patrick Maroney
    President
    Integrated Networking Technologies, Inc.
    Desk: (856)983-0001
    Cell: (609)841-5104
    Email: pmaroney@specere.org

     



    _____________________________
    From: Eric Burger < eric.burger@georgetown.edu >
    Sent: Wednesday, May 4, 2016 11:50 AM
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    To: < cti-stix@lists.oasis-open.org >


    Bret - Thanks for pinging me directly. I’ve been distracted, but I’m baaaaack .


     


    My observations:


    What Patrick M/P Patrick [I could not resist] point out is this deterministic UUID generation algorithm is (1) useful, (2) known useful
    because it is being used, and (3) is easy to implement, because it is being used.


     


    What I would also observe is the sentiment that "we can have an opaque UUID and a deterministic UUID and if people want to or eventually
    discover the usefulness of a deterministic UUID in the future" is flawed. Unless people who do not think they
    need  a deterministic UUID generator happen  to use the deterministic UUID generation algorithm, old STIX databases will not be compatible with a new, post-deterministic UUID regime.


     


    Now the true academic answer is to not have a UUID at all, because people will get it wrong and any time there is redundant data in a message
    there will be the opportunity to screw it up.


     


    Now the practitioner in me says we should have a deterministic UUID because experience is showing it is useful in practice and
    it simplifies STIX implementations  by have one and only one  way of doing it. If you don’t think you need a deterministic UUID generator, think of it as you are
    getting a UUID generator for free - you won’t have to think about it.


     



    On May 4, 2016, at 3:08 AM, Patrick Maroney < Pmaroney@specere.org > wrote:

     




    re: "So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why"."
      


     



    The 'debate' over Timestamps began in November 2013 and centered on the following identified capabilities gap and proposed solution:


     



     (1) Our CTI lexicon MUST allow representation of events at the timescales/periodicity of the operational domain we (and our machines) are
    observing, measuring, and conveying to others.     


     


    (2) We should adopt RFC 3339 for fractional seconds representation  [time-secfrac = "." 1*DIGIT]



     





    The requirements for fractional seconds were self-evident to anyone collecting, correlating, and analyzing Cyber Threat Intelligence in
    November 2013:  For example 10GbE (10×10^9 or 10 billion bits per second) was widely deployed across distributed enterprise backbones/server farms. Packet Capture and Netflow Collection were integral to our Security Operations and CSIRT Investigations.




     

    Community members provided real world use cases and challenges where (1) "My" products operate at these speeds, (2) "My" use cases require
    sub-millisecond Timestamps, etc.   Paraphrasing the arguments made against the requested change included (1) "My" product doesn't operate at these speeds, (2) "My" use cases don't require sub-millisecond Timestamps, (3) I'm using epoch time in my application,
    so I don't see the need for fractional seconds, (4) We're not going to pay attention to 'fringe' cases  



     


     


    "This is the wrong way to go and here is why":  
     


     



      A small faction should not be able to summarily reject a stated community open requirement on the basis of "we don't have or understand
    this requirement". 



     


     


     




    re: "We have debated this issue almost as long as we have debated timestamps."


     



    No, we have not even engaged in an open, inclusive discourse on this proposal.  I've responded before that the core assumptions you are
    making to the list as the basis for rejecting the concept are not accurate.   Paul Patrick has gotten it right.  


     


    The proposed concept is that the Deterministic UUID is generated using RFC 4122 Version 5 hash of the (1) Namespace, (2) Object Type, and
    (3)  Immutable properties of the Object   .   So for an Object describing the IP Address "1.2.3.4", the  Immutable property of the Object = "1.2.3.4" .    The  "1.2.3.4"  value would NEVER change for this Object.   Any changes
    to an objects immutable properties would be a new object, not a new version.


     


    I'm not going to rehash the rest...


     


    Bottom Line:


     



    Current language 


     










    "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type]
    is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID)."


     




     


    "This is the wrong way to go and here is why":  
     


     



     




    This language unnecessarily and arbitrarily constrains the method for the generation of the RFC 4122 variant UUID to Version 4 (random/pseudo
    random).


     


    We "get" that the proposed adoption of Deterministic vs. Random IDs has been rejected.   However, there's no reason to prevent those of
    us who have valid Use Cases for Deterministic IDs as part of our internal implementation details.  They will be indistinguishable from any other UUID except for the most significant 4 bits of the time stamp will be a "5" instead of a "4".



     


     



    Proposed Language





     


    "An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUID], where [object-type]
    is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant UUID. "


     





    Vote it up or down...


     



     



    Office:   (856)983-0001


    Cell:       (609)841-5104



     


    <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[20].png>


     


    President


    Integrated Networking Technologies, Inc.


    PO Box 569


    Marlton, NJ 08053





     


    From:
    < cti-stix@lists.oasis-open.org > on behalf of Bret Jordan < bret.jordan@bluecoat.com >
    Date: Tuesday, May 3, 2016 at 3:09 PM
    To: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >, Eric Burger < Eric.Burger@georgetown.edu >
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties


     



    I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are
    hashing all of the relevant fields, of which versioning would be part of.  


     


    We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between "I just do
    not like it" and "this is the wrong way to go and here is why".  If either the timestamps or IDs that we have proposed are wrong, or are going to make life miserable, please speak up and explain why. 

     









    Maybe our resident academic can chime in here?  


    Thanks,


     


    Bret



     


     


     



    Bret Jordan CISSP


    Director of Security Architecture and Standards Office of the CTO


    Blue Coat Systems



    PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE
    7415 0050


    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 









     



    On May 3, 2016, at 11:04, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote:

     



    What would be the proposed third value used in the tuple that would not break versioning?

    -
    Jason Keirstead
    STSM, Product Architect, Security Intelligence, IBM Security Systems
    www.ibm.com/security
    www.securityintelligence.com

    Without data, all you are is just another person with an opinion - Unknown


    <graycol.gif> Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo

    From:
    Paul Patrick < ppatrick@isightpartners.com >
    To:
    "Jordan, Bret" < bret.jordan@bluecoat.com >, Patrick Maroney < Pmaroney@Specere.org >
    Cc:
    "Wunder, John A." < jwunder@mitre.org >, " cti-stix@lists.oasis-open.org "
    < cti-stix@lists.oasis-open.org >
    Date:
    05/03/2016 01:37 PM
    Subject:
    Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    Sent by:
    < cti-stix@lists.oasis-open.org >






    Folks,

    I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments correctly, there seemed
    to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email.

    In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly the possibility of collisions
    that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of
    breaking relationships.

    From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support. In addition, the generated identifiers
    are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4
    algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless.

    So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used.

    I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes.


    Paul Patrick

    From:
    < cti-stix@lists.oasis-open.org >
    on behalf of "Jordan, Bret" < bret.jordan@bluecoat.com >
    Date: Monday, May 2, 2016 at 6:01 PM
    To: Patrick Maroney < Pmaroney@Specere.org >
    Cc: "Wunder, John A." < jwunder@mitre.org >,
    " cti-stix@lists.oasis-open.org "
    < cti-stix@lists.oasis-open.org >
    Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
    Resent-From: < Paul.Patrick@FireEye.com >
    Pat,


    Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates),
    your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break.

    Thanks,

    Bret



    Bret Jordan CISSP
    Director of Security Architecture and Standards Office of the CTO
    Blue Coat Systems
    PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE
    7415 0050
    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
    On Apr 30, 2016, at 13:27, Patrick Maroney < Pmaroney@Specere.org > wrote:

    re: . " This
    is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections
    on Wednesday morning."
    I have added notional changes to
    Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method.

    Format and language are notional and will need corrections by the editors.

    Basis:
    If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation
    -- They can!!!


    if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate
    and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc.
    -- They can To!!!

    The method of generation of any UUID can of course be determined from the generated UUID.
    -- Win-Win for both camps and with no cost that I can discern





    4.5.? Identifier
    Type Name: identifier
    Status: Review
    MVP: Yes

    An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type
    field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

    4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

    4.5.1.?2 Examples
    {
    "type": "indicator",
    "id": "indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836",
    ...
    }

    4.5.2 ?Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting.

    Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue)

    PublIcNameSpace = NameSpace[.subNameSpace]

    ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace)


    (4.5.2.1) Examples

    NameSpace = ' specere.org '
    subNameSpace = 'isac-2'
    ObjectClass = 'AddressObject'
    ObjectType = 'ipv4-addr'
    ObjectValue = '1.2.3.5'

    AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7

    Demonstrate deterministic generation of CTI Object IDs
    Generates Namespace Prefix deterministic generation of CTI Object IDs

    Type:AddressObjectType.ipv4-addr


    Value:1.2.3.4


    Namespace:
    development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69


    Namespace:
    production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69


    Namespace:
    operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10


    Namespace:
    isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26


    Namespace:
    isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30



    Type:AddressObjectType.ipv4-addr


    Value:1.2.3.5


    Namespace:
    development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83


    Namespace:
    production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0


    Namespace:
    operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f


    Namespace:
    isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48


    Namespace:
    isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7



    Type:DomainNameType.TLD


    Value: badguys.com

    Namespace:
    development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da


    Namespace:
    production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc


    Namespace:
    operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3


    Namespace:
    isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f


    Namespace:
    isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac



    Type:URIObjectType.URL


    Value: http://www.badguys.com/kickme?Hard

    Namespace:
    development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e


    Namespace:
    production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe


    Namespace:
    operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a


    Namespace:
    isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03


    Namespace:
    isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2



    Type:pattern


    Value:
    "pattern": { "type": "twigs", "properties": [
    ],
    "prop1": {
    }, "prop2": {
    }, "prop3": {
    "key":"FileObject:hashes/hash/simple_hash_value", "operator":"equals", "value":"c38862b4835729d979e7940d72a48172"
    "key":"FileObject:file_name", "operator":"contains", "value":"abcd.dll"
    "key":"WinRegistryKeyObject:key",
    "operator":"equals", "value":".DEFAULTSoftwareMicrosoftWindowsCurrentVersionExplorer{19127AD2­394B­70F5 ­C650­B97867BAA1F7"
    }, "prop4": {
    } "prop5": {
    }
    "key":"WinRegistryKeyObject:hive", "operator":"equals", "value":"HKEY_USERS"
    "key":"IPv4AddressObject:hive", "operator":"equals", "value":"1.2.3.4"
    "condition": "(prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS" }


    Namespace:
    development.specere.org
    Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
    ID Suffix: f7634169-2a18-5971-a919-585a4a913b72


    Namespace:
    production.specere.org
    Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
    ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56


    Namespace:
    operational-testing.specere.org
    Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
    ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a


    Namespace:
    isac-1.specere.org
    Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
    ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc


    Namespace:
    isac-2.specere.org
    Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
    ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8







    4.1.3. Version

    The version number is in the most significant 4 bits of the time
    stamp (bits 4 through 7 of the time_hi_and_version field).

    The following table lists the currently-defined versions for this
    UUID variant.

    Msb0 Msb1 Msb2 Msb3 Version Description

    0 0 0 1 1 The time-based version
    specified in this document.

    0 0 1 0 2 DCE Security version, with
    embedded POSIX UIDs.


    0 0 1 1 3 The name-based version
    specified in this document
    that uses MD5 hashing.

    0 1 0 0 4 The randomly or pseudo-
    randomly generated version
    specified in this document.

    0 1 0 1 5 The name-based version
    specified in this document
    that uses SHA-1 hashing.

    The version is more accurately a sub-type; again, we retain the term
    for compatibility.
    Patrick Maroney
    Office:
    (856)983-0001
    Cell:
    (609)841-5104

    <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png>

    President
    Integrated Networking Technologies, Inc.
    PO Box 569
    Marlton, NJ 08053

    From:
    < cti-stix@lists.oasis-open.org >
    on behalf of John Wunder < jwunder@mitre.org >
    Date: Friday, April 29, 2016 at 4:23 PM
    To: " cti-stix@lists.oasis-open.org "
    < cti-stix@lists.oasis-open.org >
    Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties

    All,

    As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content
    in the specifications through informal consensus, to review, to motions to approve the text.

    In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process:

    1.        
    Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development)

    2.        
    Normative text is developed, iterated, and settles down (Status = Development)

    3.        
    We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review)

    4.        
    After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is.

    5.        
    We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this
    will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is
    passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft)
    Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier
    phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure
    that we have consensus at a more granular level as we move forward.

    We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic.

    For this first round, please review the following sections in the STIX 2.0 specification:

    Identifier:
    https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q
    Timestamp:
    https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf
    Timestamp Precision:
    https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j
    Custom Properties:
    https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86

    This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections
    on Wednesday morning.

    Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time.

    John

     
     




     







     


     









     









  • 15.  Re: [cti-stix] Deterministic IDs - pas de deux

    Posted 05-06-2016 17:34
    Mea culpa . I forgot UUID’s have version numbers inside them . They are not entirely opaque. Go ahead. Define it as opaque. If someone happens to use a different algorithm to generate them. Great. I would then get on my high academic horse and say “compare UUID’s at your own risk. On May 4, 2016, at 2:10 PM, Piazza, Rich < rpiazza@MITRE.ORG > wrote: I think this might be a reasonable compromise – because Pat states:    (3) One can makes inferences from a Version 5 UUID in a Community of Trust where Source Namespaces used to sign and objects are shared.   Otherwise one can infer nothing else from a Version 4 or Version 5 UUID.   If you are in a community of trust you would know that you should use the optional property for whatever purpose your community has agreed to.  Others outside that community would ignore that property.      Personally, I think the same could be true if UUID 5s could be used in the id field itself, but since people seem to think that might cause problems – this could work for everyone.   One more thing, Pat – does every TLO have immutable properties?   As John mentioned, you are referring to CybOX objects, so it might be true for them, but for STIX objects, may not. CybOX btw, could do something different as far as ids go…     From:   cti-stix@lists.oasis-open.org   [ mailto:cti-stix@lists.oasis-open.org ]   On Behalf Of   Allan Thomson Sent:   Wednesday, May 04, 2016 1:44 PM To:   Jordan, Bret < bret.jordan@bluecoat.com >; Eric Burger < Eric.Burger@georgetown.edu > Cc:   Wunder, John A. < jwunder@mitre.org >;   cti-stix@lists.oasis-open.org Subject:   Re: [cti-stix] Deterministic IDs - pas de deux   Why not have a required identifier based on UUID4 (my preference) and an if vendors want a deterministic id then they can include an optional attribute for that. For systems that want to use that optional identifier it would provide support for.   allan Attachment: signature.asc Description: Message signed with OpenPGP using GPGMail