>I’m not sure that a lot of tools will need to retrieve the full list for custom vocabularies anyway. Some tools just using them for correlation and bucketing will just use whatever values they get regardless of vocabulary, others will bucket based on
>what values they get from value+vocab_uri (without retrieving the whole list), and others will share the list at design time and the reference will be more of a URI than a URL.
[sean]I would agree with this
>I would worry about the complexity of a solution where we have to send a full vocabulary in content (do you *always* send the vocabulary? if not, what do you do if you don’t already have it? how do you update a vocabulary that you sent previously?)
[sean]I also agree with this
[sean]On the Bret’s proposed idea of a layered approach I could see where this may be feasible and useful
I think it is important we consider the four identified usage scenarios from the community that drove the current vocabulary approach and ensure that the new approach
covers the necessary ground.
The four usage scenarios are:
Desire to convey a value that is conformant with a CTI community consensus vocabulary for that property and to support enforced validation that it does conform Desire to convey a value that is conformant with a consensus vocabulary (defined by other than the CTI community) for that property and to support enforced validation
that it does conform Desire to convey a value that is conformant with a consensus vocabulary (defined by other than the CTI community) for that property but not requiring support for
enforced validation that it does conform (typically due to technical, policy or political reasons) Desire to convey a value relevant to the producers context but not necessarily conformant to any available consensus vocabulary (I.e., a value that needs to be conveyed but for which no consensus vocabulary exists)
The order of the list is the clear order of preference in which approach to take. Always try to do #1. If you can’t do #1
then try to do #2. If you can’t do #2 then try to do #3. If you can’t do #1, #2 or #3 then #4 is your last resort escape clause. This allows us to drive towards interoperability first at a universal level
then at a localized level all while not ignoring that real-world use will also still have (hopefully rare) exceptions to these.
It looks like the proposed layered approach should be able to support #1 with its primary type field and an explicit default CTI community consensus vocabulary.
It looks like it should be able to support #2 with its sub_type structure as long as the consumer can access the referenced vocabulary and it is explicitly specified. This approach would put the onus on the consumer for performing any validation though,
which may likely be true for non-XML serializations of the current approach as well.
It looks like it should be able to support #3 with its sub_type structure. This actually looks very similar to the current
approach in supporting this usage scenario.
It looks like it should be able to support #4 with its sub-type structure as long as the default CTI community consensus vocabulary contained something like an “Other” value and the sub_type/vocab property were optional.
I think the only thing that really give me pause is considering whether potential values for #2, #3 or #4 would always align with some value in the default
CTI community consensus vocabulary as is described as part of the intent h ere ("fall back to something that is "close enough" for what they need”). I would suspect that even if we carefully crafted a default vocabulary that successfully covers most
of the ground, the diversity of uses out there will always have exceptions. I would think the inclusion of something like “Other” in the default vocab would mitigate this issue. Basically, try to map your non-default vocab value to the closest thing possible
in the default vocab and then provide your specific value in addition. If it is not practical to map your non-default vocab value to the closest thing possible in the default vocab then choose “Other” as the primary type and provide your specific
value in addition.
Overall, it looks workable.
sean
From: "
cti-stix@lists.oasis-open.org " <
cti-stix@lists.oasis-open.org > on behalf of John Wunder <
jwunder@mitre.org >
Date: Wednesday, October 28, 2015 at 8:57 AM
To: "
cti-stix@lists.oasis-open.org " <
cti-stix@lists.oasis-open.org >
Subject: Re: [cti-stix] [cti-users] Indicator Type / Vocabulary Implementation Questions
I’m not sure that a lot of tools will need to retrieve the full list for custom vocabularies anyway. Some tools just using them for correlation and bucketing will just use whatever values they get regardless of vocabulary, others will bucket based
on what values they get from value+vocab_uri (without retrieving the whole list), and others will share the list at design time and the reference will be more of a URI than a URL.
I really like this approach. To me it strikes a good balance between compatibility and extensibility. I would worry about the complexity of a solution where we have to send a full vocabulary in content (do you *always* send the vocabulary? if
not, what do you do if you don’t already have it? how do you update a vocabulary that you sent previously?)
John
On Oct 27, 2015, at 5:54 PM, Jordan, Bret <
bret.jordan@BLUECOAT.COM > wrote:
Agreed. But for those that really want to use their own list, and are just adamant about it, then they can publish their list on a web site or github or where ever. The key is to have at least a fall back to a value that is "close enough" for tools that have
no access to the extended vocabulary.
Now along these lines, it might be possible to send a vocabulary in a STIX package, if someone was so inclined. And the vocab field would just contain an ID REF.
I guess another hard question we really need to ask is, what is the scope of the problem we are trying to solve?
How many users are going to need this solution?
a) If the number is 0-5% maybe we do not do it until STIX 2.5 / STIX 3.0
b) If the number is between 5-20% then we should figure out a solution
c) If between 20%-40% maybe we need 2-3 defined vocabularies
c) If greater than 40%, then we need to fix the default vocabulary so they do not need to use their own.
Thanks,
Bret
Bret Jordan CISSP
Director of Security Architecture and Standards Office of the CTO
Blue Coat Systems
PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
On Oct 27, 2015, at 14:14, Jason Keirstead <
Jason.Keirstead@ca.ibm.com > wrote:
RE:
"sub_type" : {
"vocab": "
http://a.b.com/vocab-foo ",
"type": "X-RAT-Downloader"
},
The problems with this are the same as with the current system...
- As a tool, I likely do not have any internet access to publish a vocabulary list
- Even if I did, I probably do not have access to a location on the corporate website to publish it.
Sent from IBM Verse
Terry MacDonald --- [cti-stix] RE: [cti-users] Indicator Type / Vocabulary Implementation Questions ---
From:
"Terry MacDonald" <
terry@soltra.com >
To:
"Michael Hammer" <
michael.hammer@yaanatech.com >,
athiasjerome@gmail.com ,
"Jason Keirstead" <
Jason.Keirstead@ca.ibm.com >
Cc:
joep@eclecticiq.com ,
Bernd.Grobauer@siemens.com ,
Cliff.Palmer@gd-ms.com ,
cti-stix@lists.oasis-open.org ,
cti-users@lists.oasis-open.org ,
jmg@newcontext.com ,
jwunder@mitre.org ,
sbarnum@mitre.org Date:
Tue, Oct 27, 2015 1:15 PM
Subject:
[cti-stix] RE: [cti-users] Indicator Type / Vocabulary Implementation Questions
Michael,
Never apologize for bringing an opinion to the table J .
It’s the discussions and reasoning from many different viewpoints that will make STIX/TAXII and CybOX better – so throw out what ever you would like to J .
This similar idea was brought up on the CTI-STIX mailing list (where the discussion has migrated to) by Bret Jordan:
Perhaps this issue could be solved by a layered approach.....
Step 1: Greatly increase the default vocabulary, removing weirdness and things that are duplicate in nature. Try and get the default vocabulary to a 60/40 or 70/30 rule. It would also be really nice if the terms were backed by a numerical element. String
matching in code in notoriously inefficient.
Step 2: Provide a secondary entry point for an additional vocabulary for those groups that want or need to define their own. We would obviously need an entry point in to the controlled vocabulary that can be a "fall back", something that is a lot higher up
the food chain.
By doing something like this, software that can only work with the default vocabulary can skip or throw away things it does not understand and have a fall back to something that is "close enough" for what they need. In the second example below, the "sub_type"
would be options and parsers could easily throw it away or skip it. In fact most JSON parsers today naturally just skip things that do not map to a struct.
{
"stixtype": "indicator",
"type": "IP Watchlist",
....
{
or
{
"stixtype": "indicator",
"type": "Malware",
"sub_type" : {
"vocab": "
http://a.b.com/vocab-foo ",
"type": "X-RAT-Downloader"
},
....
{
It seems to me that this is a good in-between option.
I’d definitely suggest following the discussion on the CTI-STIX list. We’d really appreciate your involvement in the STIX Sub Committee if you have the time!
Cheers
Terry MacDonald
Senior STIX Subject Matter Expert
SOLTRA An FS-ISAC and DTCC Company
+61 (407) 203 206
terry@soltra.com From:
cti-users@lists.oasis-open.org [ mailto:
cti-users@lists.oasis-open.org ] On
Behalf Of Michael Hammer
Sent: Wednesday, 28 October 2015 6:57 AM
To:
athiasjerome@gmail.com ;
Jason.Keirstead@ca.ibm.com Cc:
joep@eclecticiq.com ;
Bernd.Grobauer@siemens.com ;
Cliff.Palmer@gd-ms.com ;
cti-stix@lists.oasis-open.org ;
cti-users@lists.oasis-open.org ;
jmg@newcontext.com ;
jwunder@mitre.org ;
sbarnum@mitre.org Subject: RE: [cti-users] Indicator Type / Vocabulary Implementation Questions
Pardon me if I am off base with this thought but throwing out to see what sticks.
In another group, they had similar problem of divergent lists of “vocabularies” or “dictionaries”.
Example was how many states are in a sequence of a work flow and what are they called.
Common problem seems to be that your list is not my list.
In the end, they used they used a:
DictionaryOwner: e.g. one or another standards body or proprietary (e.g. OASIS-CTI)
DictionaryAttribute: e.g. the variable name (e.g. IndicatorType)
DictionaryValue: e.g. the variable value (of form string)
Where the Owner provided the context for what permissible Values could be used.
The hope in some cases was that a sufficiently well-defined standard set could obviate the need for proprietary ones.
Regularly updating the standard, could help minimize the other lists.
However, if a set needed to be “these and only these values”, then the need for multiple sets remains.
________________________________
Michael Hammer
Principal Engineer
michael.hammer@yaanatech.com Mobile: +1 408
202 9291
542 Gibraltar Drive
Milpitas, CA 95035 USA
www.yaanatech.com From:
cti-users@lists.oasis-open.org [ mailto:
cti-users@lists.oasis-open.org ] On
Behalf Of Jerome Athias
Sent: Saturday, October 24, 2015 9:19 AM
To: Jason Keirstead <
Jason.Keirstead@ca.ibm.com >
Cc: Joep Gommers <
joep@eclecticiq.com >; Grobauer, Bernd <
Bernd.Grobauer@siemens.com >;
Cliff.Palmer@gd-ms.com ;
cti-stix@lists.oasis-open.org ;
cti-users@lists.oasis-open.org ;
John-Mark Gurney <
jmg@newcontext.com >; Wunder, John A. <
jwunder@mitre.org >;
Barnum, Sean D. <
sbarnum@mitre.org >
Subject: Re: [cti-users] Indicator Type / Vocabulary Implementation Questions
What you're pointing there is another thing I thought about which is the relationships between different vocabularies.
While partially implemented in my tools in an effort to avoid inconstancy, it was not yet introduced in the mailinglists
(Again, a structure à la CWE/CAPEC could be mid-term solution before a RDF/OWL ontology)
On Saturday, 24 October 2015, Jason Keirstead <
Jason.Keirstead@ca.ibm.com > wrote:
Should be pretty self-explanatory....
Anomalous Activity <Could be any but usually Reconnaissance/Weaponization/Delivery>
Malicious Activity <Delivery / Exploitation>
Command and Control <Command and Control>
Anonymization <Actions>
Data Exfiltration <Actions>
Lateral Movement <Installation>
Privilege Escalation <Installation>
Reconnaissance <Reconnaissance >
Host/Process Compromise <Installation>
Watchlist <N/A>
Quantified Risk <N/A>
Policy Violation ** <N/A>
-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown
Joep Gommers ---2015/10/24 06:33:42 AM---Jason, How would you feel this relates to killchain?
From: Joep Gommers <
joep@eclecticiq.com >
To: Jason Keirstead/CanEast/IBM@IBMCA
Cc: John-Mark Gurney <
jmg@newcontext.com >,
"Barnum, Sean D." <
sbarnum@mitre.org >, "Grobauer, Bernd" <
Bernd.Grobauer@siemens.com >,
"Wunder, John A." <
jwunder@mitre.org >, "
Cliff.Palmer@gd-ms.com "
<
Cliff.Palmer@gd-ms.com >, "
cti-users@lists.oasis-open.org "
<
cti-users@lists.oasis-open.org >, "
cti-stix@lists.oasis-open.org "
<
cti-stix@lists.oasis-open.org >
Date: 2015/10/24 06:33 AM
Subject: Re: [cti-users] Re: [cti-stix] Re: [cti-users] Indicator Type
/ Vocabulary Implementation Questions
Jason,
How would you feel this relates to killchain?
J
Sent from my iPhone
On 24 Oct 2015, at 11:05, Jason Keirstead <
Jason.Keirstead@ca.ibm.com >
wrote:
I like the direction this is going
" Removing type information would reduce the IndicatorTypeVocab down to:
Compromised
Malicious
Watchlist
C2
Anonymization
Exfiltration
"
This is very similar to what I have been working through
This was my internal list so far - thoughts?
Anomalous Activity
Malicious Activity
Command and Control *
Anonymization
Data Exfiltration
Lateral Movement
Privilege Escalation
Reconnaissance
Host/Process Compromise
Watchlist
Quantified Risk
Policy Violation **
* I prefer descriptive names other than acronyms like "C2", it makes it easier for translation purposes.
** Not sure about this one... its kind of straying outside the CTI realm.. although i do see a great value / need for it in the vocabulary.
-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown
<graycol.gif> John-Mark Gurney ---2015/10/23 03:57:30 PM---I have created an issue for this as when I was reviewing the vocab list, it did not cover our use ca
From: John-Mark Gurney <
jmg@newcontext.com >
To: "Barnum, Sean D." <
sbarnum@mitre.org >
Cc: "Grobauer, Bernd" <
Bernd.Grobauer@siemens.com >, "Wunder, John A."
<
jwunder@mitre.org >, Jason Keirstead/CanEast/IBM@IBMCA, "
Cliff.Palmer@gd-ms.com "
<
Cliff.Palmer@gd-ms.com >, "
cti-users@lists.oasis-open.org "
<
cti-users@lists.oasis-open.org >,
cti-stix@lists.oasis-open.org Date: 2015/10/23 03:57 PM
Subject: [cti-stix] Re: [cti-users] Indicator Type / Vocabulary Implementation Questions
Sent by: <
cti-stix@lists.oasis-open.org >
I have created an issue for this as when I was reviewing the vocab list, it did not cover our use case.
The issue I created:
https://github.com/STIXProject/specifications/issues/35 I believe that this will help people use the Vocab better, and may reduce the need for custom vocabs.
Please comment on this issue to provide feed back.
Thanks.
I have included the text of the issue here for reference:
There is a discussion on cti-users and cti-stix about improving the IndicatorTypeVocab.
I believe that having a vocab is a useful thing. But I believe the existing vocab needs to be improved.
First off, type information, like e-mail, ip, file hash, domain, etc. should be removed. You should/must be able to get this information from the Observable that is part of the Indicator.
For one, there is no vocab to describe a malicious observiable, say network packet, stream, or other activity. Though if the e-mail type is removed from Malicious E-mail, and it just became Malicious (Observable), then we would have something.
Removing type information would reduce the IndicatorTypeVocab down to:
Compromised
Malicious
Watchlist
C2
Anonymization
Exfiltration
The first three are interesting, Compromised means that this Observable indicates that you ARE compromised. The Malicious means that you WILL be compromised by this Observable and Watchlist means that you MAY get compromised by this Observable.
Arguably, C2 should fall under Compromised, but as it probably requires further investigation to figure out the original compromised host, I'm fine leaving this as it's own separate type.
On Fri, Oct 23, 2015 at 7:19 AM, Barnum, Sean D. <
sbarnum@mitre.org >
wrote:
I think the first step would be to enter an issue in the tracker for this so that we can get it on the table. I also agree with an earlier statement that the issue of default vocab values has clear overlap
with the interoperability SC so while we need to work internally within the STIX SC for ensuring our default vocabs have the appropriate values for STIX use cases it probably also makes sense to work at a higher level on the process by which we define and
manage the various default controlled vocabs.
[attachment "graycol.gif" deleted by Jason Keirstead/CanEast/IBM] [attachment "graycol.gif" deleted by Jason Keirstead/CanEast/IBM]