OASIS Cyber Threat Intelligence (CTI) TC

  • 1.  Golang libstix2 updates

    Posted 01-10-2020 06:07
    All, I have been working on my libstix2 libraries for Golang ( https://github.com/freetaxii/libstix2 ) and have added basic support for all of the STIX 2.1 objects. In addition I have been working on adding support for custom properties. This note explains how they work. Right now I only have it working for Attack Patterns, but I will copy the code across all of the objects soon, unless someone has a better way of solving this problem. A) First, let s deal with the decoding of an attack pattern JSON string that has a custom property called some_custom_property , where that custom property is of type string. You may have an attack pattern that looks like this: { "type": "attack-pattern", "spec_version": "2.1", "id": "attack-pattern--d62e1eff-eb93-42e2-bd90-dabff3b93427", "created": "2018-06-05T18:25:15.917Z", "modified": "2018-06-05T18:25:15.917Z", "name": "Phishing", "aliases": ["Banking1", "ATM2"], "some_custom_property": "some_custom_value" } In your code you have consumed this JSON string and stored it in a []byte called foo . So you could decode it by doing: o, _ := attackpattern.Decode(foo) This will give you an attack pattern object called o and all of the custom properties will be stored in a property called o.Custom which is a map[string]*json.RawMessage. This means that we will have all of the key names for the custom properties, but all of their values will be untouched and left as a []byte. So we are not decoding the values at this stage. So if you do not know how to process them, you do not need to worry about it. Just store them as an array of bytes. If you were to print out o.Custom you would get: map[some_custom_property:[34 115 111 109 101 95 99 117 115 116 111 109 95 118 97 108 117 101 34]] Then when you want to process that data in o.Custom, if you know how to, you would just unmarshal those properties, such as: var data string json.Unmarshal(o.Custom["some_custom_property"], &data) If you were to print out value of data you would get: some_custom_value B) Second, let s deal with encoding and adding properties to the objects that I have defined in my library. You would create a new type and embed the one from my library in it, like so: type myCustomAttackPattern struct { *attackpattern.AttackPattern SomeCustomProperty string `json:"some_custom_propety,omitempty"` } Then in your main program you could do: o := attackpattern.New() customAP := myCustomAttackPattern{AttackPattern: o} or o := attackpattern.New() var customAP myCustomAttackPattern customAP.AttackPattern = o At this point you can then populate your new custom property with data and encode it like so: customAP.SomeCustomProperty = "some custom string data" data, - := json.MarshalIndent(customAP, "", " ") Now if you print out data you will get an attack pattern with your new custom property. { "type": "attack-pattern", "spec_version": "2.1", "id": "attack-pattern--2b88c3ff-410d-44c4-9056-2ead668ab13c", "created": "2020-01-10T05:52:30.494Z", "modified": "2020-01-10T05:52:30.494Z", "some_custom_propety": "some custom string data" } I have two examples in my examples directory that show this, 06-decode-custom-properties.go and 07-use-custom-properties.go Thanks, Bret PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."


  • 2.  Re: [cti] Golang libstix2 updates

    Posted 01-10-2020 13:49
    Some thoughts;   For us at least, most all usage of custom properties are complex (we have a custom property that underneath it has a JSON object with a bunch of sub properties, it's not just a string)   I am unsure why, you are saving the custom property as a byte array. Why wouldn't you just decode the JSON and save it as an object as per normal? It would make it easier to work with.    - Jason Keirstead Chief Architect - IBM Security Threat Management www.ibm.com/security Would you like me to give you a formula for success? It's quite simple, really. Double your rate of failure. - Thomas J. Watson    


  • 3.  Re: [cti] Golang libstix2 updates

    Posted 01-10-2020 16:23
    When I designed this I figured most custom properties would be complex. The use cases I came up with are: 1) You get an object without custom data. There is nothing to do and using the library is super simple. 2) You get an object with custom properties and you do not know how to process them. In addition you have a simple datastore . In this case, you can easily just store the byte array and be done with it. This way you do not lose the data, nor do you have issues trying to store the complex data. 3) You get an object with custom properties and once again you do not know how to process them. This time you do have a no-sql datastore, or something that can store native JSON. In this case, you can either store them as a byte array or easily range through the values in the map and unmarshall the complex data. This would allow you to store the custom properties as native JSON, even if they are complex. 4) You get an object with custom properties and you do know how to process them. In this case you can easily range through the values in the map and unmarshall them. This would in effect give you back the original data. My goal was to enable my library to support custom properties, even ones that are complex. If you have a better was of doing this or solving this, please let me know. Thanks, Bret PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." On Fri, Jan 10, 2020 at 6:48 AM Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: Some thoughts; For us at least, most all usage of custom properties are complex (we have a custom property that underneath it has a JSON object with a bunch of sub properties, it's not just a string) I am unsure why, you are saving the custom property as a byte array. Why wouldn't you just decode the JSON and save it as an object as per normal? It would make it easier to work with. - Jason Keirstead Chief Architect - IBM Security Threat Management www.ibm.com/security "Would you like me to give you a formula for success? It's quite simple, really. Double your rate of failure." - Thomas J. Watson


  • 4.  Re: [cti] Golang libstix2 updates

    Posted 01-10-2020 23:59
    Why do you need to worry about if it is complex?  It is Json in either simple and complex scenarios ?  If yes, then it's just json or equivalent after it is parsed. If someone wants to type the property at a later point that would be up to them. On Fri, Jan 10, 2020 at 11:24 AM -0500, Bret Jordan < bret.jordan@broadcom.com > wrote: When I designed this I figured most custom properties would be complex.  The use cases I came up with are: 1) You get an object without custom data. There is nothing to do and using the library is super simple.  2) You get an object with custom properties and you do not know how to process them. In addition you have a simple datastore . In this case, you can easily just store the byte array and be done with it.  This way you do not lose the data, nor do you have issues trying to store the complex data.  3) You get an object with custom properties and once again you do not know how to process them. This time you do have a no-sql datastore, or something that can store native JSON. In this case, you can either store them as a byte array or easily range through the values in the map and unmarshall the complex data. This would allow you to store the custom properties as native JSON, even if they are complex.    4) You get an object with custom properties and you do know how to process them. In this case you can easily range through the values in the map and unmarshall them. This would in effect give you back the original data. My goal was to enable my library to support custom properties, even ones that are complex.  If you have a better was of doing this or solving this, please let me know.  Thanks, Bret PGP Fingerprint:  63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg. On Fri, Jan 10, 2020 at 6:48 AM Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: Some thoughts;   For us at least, most all usage of custom properties are complex (we have a custom property that underneath it has a JSON object with a bunch of sub properties, it's not just a string)   I am unsure why, you are saving the custom property as a byte array. Why wouldn't you just decode the JSON and save it as an object as per normal? It would make it easier to work with.    - Jason Keirstead Chief Architect - IBM Security Threat Management www.ibm.com/security Would you like me to give you a formula for success? It's quite simple, really. Double your rate of failure. - Thomas J. Watson    


  • 5.  Re: [cti] Golang libstix2 updates

    Posted 01-11-2020 01:09
    From my library's perspective it does not matter if it is complex or a string or a number. I am just going to store it as a byte array. If I pre-parse it it will be harder for you to store and deal with if you do not know what they are because it will be stored in a map[string]interface{} object. So you would have to do type assertions on ever nested element and you may not know what those elements are, which means you would have to do a switch statement on the types and then process them all just to store the data for something you do not know how to work with. If you do not know how to deal with the custom properties having them all as a byte array makes is super simple to deal with. Keep in mind, if you know how to deal with the custom properties you have 1 more step in your code. Not a big deal IMHO. Thanks, Bret PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." On Fri, Jan 10, 2020 at 4:59 PM < sr@ctin.us > wrote: Why do you need to worry about if it is complex? It is Json in either simple and complex scenarios ? If yes, then it's just json or equivalent after it is parsed. If someone wants to type the property at a later point that would be up to them. On Fri, Jan 10, 2020 at 11:24 AM -0500, "Bret Jordan" < bret.jordan@broadcom.com > wrote: When I designed this I figured most custom properties would be complex. The use cases I came up with are: 1) You get an object without custom data. There is nothing to do and using the library is super simple. 2) You get an object with custom properties and you do not know how to process them. In addition you have a simple datastore . In this case, you can easily just store the byte array and be done with it. This way you do not lose the data, nor do you have issues trying to store the complex data. 3) You get an object with custom properties and once again you do not know how to process them. This time you do have a no-sql datastore, or something that can store native JSON. In this case, you can either store them as a byte array or easily range through the values in the map and unmarshall the complex data. This would allow you to store the custom properties as native JSON, even if they are complex. 4) You get an object with custom properties and you do know how to process them. In this case you can easily range through the values in the map and unmarshall them. This would in effect give you back the original data. My goal was to enable my library to support custom properties, even ones that are complex. If you have a better was of doing this or solving this, please let me know. Thanks, Bret PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." On Fri, Jan 10, 2020 at 6:48 AM Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: Some thoughts; For us at least, most all usage of custom properties are complex (we have a custom property that underneath it has a JSON object with a bunch of sub properties, it's not just a string) I am unsure why, you are saving the custom property as a byte array. Why wouldn't you just decode the JSON and save it as an object as per normal? It would make it easier to work with. - Jason Keirstead Chief Architect - IBM Security Threat Management www.ibm.com/security "Would you like me to give you a formula for success? It's quite simple, really. Double your rate of failure." - Thomas J. Watson


  • 6.  Re: [cti] Golang libstix2 updates

    Posted 01-11-2020 14:03
    Here is a 2.x written in Go that s being actively developed by someone who works for me in his spare time. https://github.com/TcM1911/stix2 Best Regards,  Nicholas Hayden, CISSP, GICSP, Sec+ Head of Global Intelligence Operations Senior Director of Threat Intelligence Anomali   anomali.com 808 Winslow St Redwood City, CA 94063 Phone: (207) 404-1435  Twitter: @anomali On Jan 10, 2020, at 8:09 PM, Bret Jordan <bret.jordan@broadcom.com> wrote: ï From my library's perspective it does not matter if it is complex or a string or a number.  I am just going to store it as a byte array.  If I pre-parse it it will be harder for you to store and deal with if you do not know what they are because it will be stored in a map[string]interface{} object.  So you would have to do type assertions on ever nested element and you may not know what those elements are, which means you would have to do a switch statement on the types and then process them all just to store the data for something you do not know how to work with.  If you do not know how to deal with the custom properties having them all as a byte array makes is super simple to deal with. Keep in mind, if you know how to deal with the custom properties you have 1 more step in your code. Not a big deal IMHO.  Thanks, Bret PGP Fingerprint:  63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg. On Fri, Jan 10, 2020 at 4:59 PM < sr@ctin.us > wrote: Why do you need to worry about if it is complex?  It is Json in either simple and complex scenarios ?  If yes, then it's just json or equivalent after it is parsed. If someone wants to type the property at a later point that would be up to them. On Fri, Jan 10, 2020 at 11:24 AM -0500, Bret Jordan < bret.jordan@broadcom.com > wrote: When I designed this I figured most custom properties would be complex.  The use cases I came up with are: 1) You get an object without custom data. There is nothing to do and using the library is super simple.  2) You get an object with custom properties and you do not know how to process them. In addition you have a simple datastore . In this case, you can easily just store the byte array and be done with it.  This way you do not lose the data, nor do you have issues trying to store the complex data.  3) You get an object with custom properties and once again you do not know how to process them. This time you do have a no-sql datastore, or something that can store native JSON. In this case, you can either store them as a byte array or easily range through the values in the map and unmarshall the complex data. This would allow you to store the custom properties as native JSON, even if they are complex.    4) You get an object with custom properties and you do know how to process them. In this case you can easily range through the values in the map and unmarshall them. This would in effect give you back the original data. My goal was to enable my library to support custom properties, even ones that are complex.  If you have a better was of doing this or solving this, please let me know.  Thanks, Bret PGP Fingerprint:  63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050 Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg. On Fri, Jan 10, 2020 at 6:48 AM Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote: Some thoughts;   For us at least, most all usage of custom properties are complex (we have a custom property that underneath it has a JSON object with a bunch of sub properties, it's not just a string)   I am unsure why, you are saving the custom property as a byte array. Why wouldn't you just decode the JSON and save it as an object as per normal? It would make it easier to work with.    - Jason Keirstead Chief Architect - IBM Security Threat Management www.ibm.com/security Would you like me to give you a formula for success? It's quite simple, really. Double your rate of failure. - Thomas J. Watson