CTI STIX Subcommittee

Dear All, I’d like to ask for your opinion. Use-case; many producers create intelligence with widely different content (names/meta/context) for the same threat information. Additionally, many producer don’t re-use or across producers we don’t re-use STIX IDs. Therefor, the challenge of duplication is significant.  While we’ve already have many non-STIX way of dealing with this at EclecticIQ, I wonder if STIX versioning idioms aren’t a way to accomplish part of this. Example; Before: TTP A: Zeus, version 1 – namespace vendorA TTP B: Zeus, version 1 – namespace vendorB TTP C: Zeus, version 1 – namespace vendorC After: TTP D: Zeus version 1 – my own namespace Related TTPs Supersedes IDREF TTP A version 1 Supersedes IDREF TTP B version 1 Supersedes IDREF TTP C version 1 Basically telling my STIX authority that TTP A/B/C version 1 no longer should be current and then TTP D version 1 (my analytic decision that my Zeus is now all other Zeus) is actually analytically equal to the others? J-

While not in an sql world (deduplication), The Relationship object will help On Thursday, 29 October 2015, Joep Gommers < joep@eclecticiq.com > wrote: Dear All, I’d like to ask for your opinion. Use-case; many producers create intelligence with widely different content (names/meta/context) for the same threat information. Additionally, many producer don’t re-use or across producers we don’t re-use STIX IDs. Therefor, the challenge of duplication is significant.  While we’ve already have many non-STIX way of dealing with this at EclecticIQ, I wonder if STIX versioning idioms aren’t a way to accomplish part of this. Example; Before: TTP A: Zeus, version 1 – namespace vendorA TTP B: Zeus, version 1 – namespace vendorB TTP C: Zeus, version 1 – namespace vendorC After: TTP D: Zeus version 1 – my own namespace Related TTPs Supersedes IDREF TTP A version 1 Supersedes IDREF TTP B version 1 Supersedes IDREF TTP C version 1 Basically telling my STIX authority that TTP A/B/C version 1 no longer should be current and then TTP D version 1 (my analytic decision that my Zeus is now all other Zeus) is actually analytically equal to the others? J-

There are two related threads, but will insert here.


 See: [cti-stix] Proposal to establish Sightings (#306) and Relationships (#291) as our official issue topics under active consideration for STIX v2.0]



Propose that a standard that creates the UUID using a collision avoidance One Way Hash derived from the Source Namespace and Indicator value would solve the many issues around duplication, deconfliction,  fidelity, and "parroting"
in general.


This also provides methods for Non-Attributional Source Pathway Traceability.  If I am an aggregator, I would strip off the upstream Namespace prefix (when/if required to strip attribution) and pass on the unaltered Source
Unique UUID with my namespace prefix.  This provides the method to trace back the TLO to the next upstream provider who in turn can reference the UUID to trace back to the next "hop" until the original Source is reached.  





This allows one to pass on their own sightings, analysis findings, enrichment (and if this includes originally sourced TLOs that relate to the TLO Sourced elsewhere, these assertions and relations).


This also provides for reconciliation of multiple independently derived/sourced assertions of the same TLOs and downstream correlation to derive a high fidelity holistic global view of the related TTPs and how they change
overtime.


There is only a one time performance cost for the Hashing function at the time of original TLO creation.  Therefore a strong low collision hashing algorithm could be selected.





Patrick Maroney









From: < cti-stix@lists.oasis-open.org > on behalf of Jerome Athias < athiasjerome@gmail.com >
Date: Thursday, October 29, 2015 at 8:37 AM
To: Joep Gommers < joep@eclecticiq.com >
Cc: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org >
Subject: Re: [cti-stix] STIX versioning as an interim solution to deduplication




While not in an sql world (deduplication),
The Relationship object will help

On Thursday, 29 October 2015, Joep Gommers < joep@eclecticiq.com > wrote:


Dear All,


I’d like to ask for your opinion.


Use-case; many producers create intelligence with widely different content (names/meta/context) for the same threat information. Additionally, many producer don’t re-use or across producers we don’t re-use STIX IDs. Therefor, the challenge of duplication
is significant. 


While we’ve already have many non-STIX way of dealing with this at EclecticIQ, I wonder if STIX versioning idioms aren’t a way to accomplish part of this.


Example;


Before:
TTP A: Zeus, version 1 – namespace vendorA
TTP B: Zeus, version 1 – namespace vendorB
TTP C: Zeus, version 1 – namespace vendorC


After:
TTP D: Zeus version 1 – my own namespace
Related TTPs
Supersedes IDREF TTP A version 1
Supersedes IDREF TTP B version 1
Supersedes IDREF TTP C version 1


Basically telling my STIX authority that TTP A/B/C version 1 no longer should be current and then TTP D version 1 (my analytic decision that my Zeus is now all other Zeus) is actually analytically equal to the others?


J-

No UUIDs à la CPE? On Thursday, 29 October 2015, Patrick Maroney < Pmaroney@specere.org > wrote: There are two related threads, but will insert here.  See: [cti-stix] Proposal to establish Sightings (#306) and Relationships (#291) as our official issue topics under active consideration for STIX v2.0] Propose that a standard that creates the UUID using a collision avoidance One Way Hash derived from the Source Namespace and Indicator value would solve the many issues around duplication, deconfliction,  fidelity, and "parroting" in general. This also provides methods for Non-Attributional Source Pathway Traceability.  If I am an aggregator, I would strip off the upstream Namespace prefix (when/if required to strip attribution) and pass on the unaltered Source Unique UUID with my namespace prefix.  This provides the method to trace back the TLO to the next upstream provider who in turn can reference the UUID to trace back to the next "hop" until the original Source is reached.   This allows one to pass on their own sightings, analysis findings, enrichment (and if this includes originally sourced TLOs that relate to the TLO Sourced elsewhere, these assertions and relations). This also provides for reconciliation of multiple independently derived/sourced assertions of the same TLOs and downstream correlation to derive a high fidelity holistic global view of the related TTPs and how they change overtime. There is only a one time performance cost for the Hashing function at the time of original TLO creation.  Therefore a strong low collision hashing algorithm could be selected. Patrick Maroney From: < cti-stix@lists.oasis-open.org > on behalf of Jerome Athias < athiasjerome@gmail.com > Date: Thursday, October 29, 2015 at 8:37 AM To: Joep Gommers < joep@eclecticiq.com > Cc: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] STIX versioning as an interim solution to deduplication While not in an sql world (deduplication), The Relationship object will help On Thursday, 29 October 2015, Joep Gommers < joep@eclecticiq.com > wrote: Dear All, I’d like to ask for your opinion. Use-case; many producers create intelligence with widely different content (names/meta/context) for the same threat information. Additionally, many producer don’t re-use or across producers we don’t re-use STIX IDs. Therefor, the challenge of duplication is significant.  While we’ve already have many non-STIX way of dealing with this at EclecticIQ, I wonder if STIX versioning idioms aren’t a way to accomplish part of this. Example; Before: TTP A: Zeus, version 1 – namespace vendorA TTP B: Zeus, version 1 – namespace vendorB TTP C: Zeus, version 1 – namespace vendorC After: TTP D: Zeus version 1 – my own namespace Related TTPs Supersedes IDREF TTP A version 1 Supersedes IDREF TTP B version 1 Supersedes IDREF TTP C version 1 Basically telling my STIX authority that TTP A/B/C version 1 no longer should be current and then TTP D version 1 (my analytic decision that my Zeus is now all other Zeus) is actually analytically equal to the others? J-

To clarify reference to the Common Platform Enumeration's UUIDs It is relative to having the version(ing) in (at the end of)  the UUID On Thursday, 29 October 2015, Jerome Athias < athiasjerome@gmail.com > wrote: No UUIDs à la CPE? On Thursday, 29 October 2015, Patrick Maroney < Pmaroney@specere.org > wrote: There are two related threads, but will insert here.  See: [cti-stix] Proposal to establish Sightings (#306) and Relationships (#291) as our official issue topics under active consideration for STIX v2.0] Propose that a standard that creates the UUID using a collision avoidance One Way Hash derived from the Source Namespace and Indicator value would solve the many issues around duplication, deconfliction,  fidelity, and "parroting" in general. This also provides methods for Non-Attributional Source Pathway Traceability.  If I am an aggregator, I would strip off the upstream Namespace prefix (when/if required to strip attribution) and pass on the unaltered Source Unique UUID with my namespace prefix.  This provides the method to trace back the TLO to the next upstream provider who in turn can reference the UUID to trace back to the next "hop" until the original Source is reached.   This allows one to pass on their own sightings, analysis findings, enrichment (and if this includes originally sourced TLOs that relate to the TLO Sourced elsewhere, these assertions and relations). This also provides for reconciliation of multiple independently derived/sourced assertions of the same TLOs and downstream correlation to derive a high fidelity holistic global view of the related TTPs and how they change overtime. There is only a one time performance cost for the Hashing function at the time of original TLO creation.  Therefore a strong low collision hashing algorithm could be selected. Patrick Maroney From: < cti-stix@lists.oasis-open.org > on behalf of Jerome Athias < athiasjerome@gmail.com > Date: Thursday, October 29, 2015 at 8:37 AM To: Joep Gommers < joep@eclecticiq.com > Cc: " cti-stix@lists.oasis-open.org " < cti-stix@lists.oasis-open.org > Subject: Re: [cti-stix] STIX versioning as an interim solution to deduplication While not in an sql world (deduplication), The Relationship object will help On Thursday, 29 October 2015, Joep Gommers < joep@eclecticiq.com > wrote: Dear All, I’d like to ask for your opinion. Use-case; many producers create intelligence with widely different content (names/meta/context) for the same threat information. Additionally, many producer don’t re-use or across producers we don’t re-use STIX IDs. Therefor, the challenge of duplication is significant.  While we’ve already have many non-STIX way of dealing with this at EclecticIQ, I wonder if STIX versioning idioms aren’t a way to accomplish part of this. Example; Before: TTP A: Zeus, version 1 – namespace vendorA TTP B: Zeus, version 1 – namespace vendorB TTP C: Zeus, version 1 – namespace vendorC After: TTP D: Zeus version 1 – my own namespace Related TTPs Supersedes IDREF TTP A version 1 Supersedes IDREF TTP B version 1 Supersedes IDREF TTP C version 1 Basically telling my STIX authority that TTP A/B/C version 1 no longer should be current and then TTP D version 1 (my analytic decision that my Zeus is now all other Zeus) is actually analytically equal to the others? J-

The problem with de-dup is harvesting all of the nuggets of extra information that are tied to the references that you are about to throw away. Bret  Sent from my Commodore 64 On Oct 29, 2015, at 5:38 AM, Jerome Athias < athiasjerome@gmail.com > wrote: While not in an sql world (deduplication), The Relationship object will help On Thursday, 29 October 2015, Joep Gommers < joep@eclecticiq.com > wrote: Dear All, I’d like to ask for your opinion. Use-case; many producers create intelligence with widely different content (names/meta/context) for the same threat information. Additionally, many producer don’t re-use or across producers we don’t re-use STIX IDs. Therefor, the challenge of duplication is significant.  While we’ve already have many non-STIX way of dealing with this at EclecticIQ, I wonder if STIX versioning idioms aren’t a way to accomplish part of this. Example; Before: TTP A: Zeus, version 1 – namespace vendorA TTP B: Zeus, version 1 – namespace vendorB TTP C: Zeus, version 1 – namespace vendorC After: TTP D: Zeus version 1 – my own namespace Related TTPs Supersedes IDREF TTP A version 1 Supersedes IDREF TTP B version 1 Supersedes IDREF TTP C version 1 Basically telling my STIX authority that TTP A/B/C version 1 no longer should be current and then TTP D version 1 (my analytic decision that my Zeus is now all other Zeus) is actually analytically equal to the others? J-

could be discussed aggregates, non-determinism and asynchrony 2015-10-29 19:55 GMT+03:00 Jordan, Bret <bret.jordan@bluecoat.com>: > The problem with de-dup is harvesting all of the nuggets of extra > information that are tied to the references that you are about to throw > away. > > Bret > > Sent from my Commodore 64 > > On Oct 29, 2015, at 5:38 AM, Jerome Athias <athiasjerome@gmail.com> wrote: > > While not in an sql world (deduplication), > The Relationship object will help > > On Thursday, 29 October 2015, Joep Gommers <joep@eclecticiq.com> wrote: >> >> Dear All, >> >> I’d like to ask for your opinion. >> >> Use-case; many producers create intelligence with widely different content >> (names/meta/context) for the same threat information. Additionally, many >> producer don’t re-use or across producers we don’t re-use STIX IDs. >> Therefor, the challenge of duplication is significant. >> >> While we’ve already have many non-STIX way of dealing with this at >> EclecticIQ, I wonder if STIX versioning idioms aren’t a way to accomplish >> part of this. >> >> Example; >> >> Before: >> TTP A: Zeus, version 1 – namespace vendorA >> TTP B: Zeus, version 1 – namespace vendorB >> TTP C: Zeus, version 1 – namespace vendorC >> >> After: >> TTP D: Zeus version 1 – my own namespace >> Related TTPs >> Supersedes IDREF TTP A version 1 >> Supersedes IDREF TTP B version 1 >> Supersedes IDREF TTP C version 1 >> >> Basically telling my STIX authority that TTP A/B/C version 1 no longer >> should be current and then TTP D version 1 (my analytic decision that my >> Zeus is now all other Zeus) is actually analytically equal to the others? >> >> J-

At this stage I think we should consider algorithms to handle identifiers We should also think about moving to an adaptive model On Thursday, 29 October 2015, Jerome Athias < athiasjerome@gmail.com > wrote: could be discussed aggregates, non-determinism and asynchrony 2015-10-29 19:55 GMT+03:00 Jordan, Bret < bret.jordan@bluecoat.com >: > The problem with de-dup is harvesting all of the nuggets of extra > information that are tied to the references that you are about to throw > away. > > Bret > > Sent from my Commodore 64 > > On Oct 29, 2015, at 5:38 AM, Jerome Athias < athiasjerome@gmail.com > wrote: > > While not in an sql world (deduplication), > The Relationship object will help > > On Thursday, 29 October 2015, Joep Gommers < joep@eclecticiq.com > wrote: >> >> Dear All, >> >> I’d like to ask for your opinion. >> >> Use-case; many producers create intelligence with widely different content >> (names/meta/context) for the same threat information. Additionally, many >> producer don’t re-use or across producers we don’t re-use STIX IDs. >> Therefor, the challenge of duplication is significant. >> >> While we’ve already have many non-STIX way of dealing with this at >> EclecticIQ, I wonder if STIX versioning idioms aren’t a way to accomplish >> part of this. >> >> Example; >> >> Before: >> TTP A: Zeus, version 1 – namespace vendorA >> TTP B: Zeus, version 1 – namespace vendorB >> TTP C: Zeus, version 1 – namespace vendorC >> >> After: >> TTP D: Zeus version 1 – my own namespace >> Related TTPs >> Supersedes IDREF TTP A version 1 >> Supersedes IDREF TTP B version 1 >> Supersedes IDREF TTP C version 1 >> >> Basically telling my STIX authority that TTP A/B/C version 1 no longer >> should be current and then TTP D version 1 (my analytic decision that my >> Zeus is now all other Zeus) is actually analytically equal to the others? >> >> J-

I’m no expert on using algorithms as identifiers are moving to an adaptive model, but I can attest to the fact that tracking UUIDs over time for home grown
content is a nightmare.  Until intelligence sources are very mature, they will face challenges with versioning.  De-duplication into “gold” objects melding all known intelligence together has been the solution we’ve been using for a while, but we haven’t tried
turning that back into STIX… we just persist this in a custom ontology.
 

Thanks,
 
Alex

 

From: cti-stix@lists.oasis-open.org [mailto:cti-stix@lists.oasis-open.org]
On Behalf Of Jerome Athias
Sent: Friday, November 13, 2015 4:33 AM
To: Jordan, Bret
Cc: Joep Gommers; cti-stix@lists.oasis-open.org
Subject: Re: [cti-stix] STIX versioning as an interim solution to deduplication

 
At this stage I think we should consider algorithms to handle identifiers

We should also think about moving to an adaptive model

On Thursday, 29 October 2015, Jerome Athias < athiasjerome@gmail.com > wrote:
could be discussed aggregates, non-determinism and asynchrony

2015-10-29 19:55 GMT+03:00 Jordan, Bret < bret.jordan@bluecoat.com >:
> The problem with de-dup is harvesting all of the nuggets of extra
> information that are tied to the references that you are about to throw
> away.
>
> Bret
>
> Sent from my Commodore 64
>
> On Oct 29, 2015, at 5:38 AM, Jerome Athias < athiasjerome@gmail.com > wrote:
>
> While not in an sql world (deduplication),
> The Relationship object will help
>
> On Thursday, 29 October 2015, Joep Gommers < joep@eclecticiq.com > wrote:
>>
>> Dear All,
>>
>> I’d like to ask for your opinion.
>>
>> Use-case; many producers create intelligence with widely different content
>> (names/meta/context) for the same threat information. Additionally, many
>> producer don’t re-use or across producers we don’t re-use STIX IDs.
>> Therefor, the challenge of duplication is significant.
>>
>> While we’ve already have many non-STIX way of dealing with this at
>> EclecticIQ, I wonder if STIX versioning idioms aren’t a way to accomplish
>> part of this.
>>
>> Example;
>>
>> Before:
>> TTP A: Zeus, version 1 – namespace vendorA
>> TTP B: Zeus, version 1 – namespace vendorB
>> TTP C: Zeus, version 1 – namespace vendorC
>>
>> After:
>> TTP D: Zeus version 1 – my own namespace
>> Related TTPs
>> Supersedes IDREF TTP A version 1
>> Supersedes IDREF TTP B version 1
>> Supersedes IDREF TTP C version 1
>>
>> Basically telling my STIX authority that TTP A/B/C version 1 no longer
>> should be current and then TTP D version 1 (my analytic decision that my
>> Zeus is now all other Zeus) is actually analytically equal to the others?
>>
>> J-


This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

That becomes hard in an automated way and can become a denial of service. I guess if there was a workflow piece that allowed someone to say that this is more accurate or has more information than something else. We see this problem in the AV sector today. One piece of malware is called something different by every vendor when in reality it is all the same thing. Bret Sent from my Commodore 64 > On Oct 29, 2015, at 2:45 AM, Joep Gommers <joep@eclecticiq.com> wrote: > > Dear All, > > I’d like to ask for your opinion. > > Use-case; many producers create intelligence with widely different content (names/meta/context) for the same threat information. Additionally, many producer don’t re-use or across producers we don’t re-use STIX IDs. Therefor, the challenge of duplication is significant. > > While we’ve already have many non-STIX way of dealing with this at EclecticIQ, I wonder if STIX versioning idioms aren’t a way to accomplish part of this. > > Example; > > Before: > TTP A: Zeus, version 1 – namespace vendorA > TTP B: Zeus, version 1 – namespace vendorB > TTP C: Zeus, version 1 – namespace vendorC > > After: > TTP D: Zeus version 1 – my own namespace > Related TTPs > Supersedes IDREF TTP A version 1 > Supersedes IDREF TTP B version 1 > Supersedes IDREF TTP C version 1 > > Basically telling my STIX authority that TTP A/B/C version 1 no longer should be current and then TTP D version 1 (my analytic decision that my Zeus is now all other Zeus) is actually analytically equal to the others? > > J-