OASIS Collaborative Automated Course of Action Operations (CACAO) for Cyber Secu

 View Only
Expand all | Collapse all

RE: [cacao] [EXT] [cacao] Remaining work items

  • 1.  RE: [cacao] [EXT] [cacao] Remaining work items

    Posted 11-18-2022 16:25




    Looks good to me.


    -Vasileios 


    On Nov 18, 2022 16:47, "Taylor, Marlon" <Marlon.Taylor@cisa.dhs.gov> wrote:



    In support of adoption, as an initial early release of the specification to be evolved, I think we can have consensus around:

    playbook_types as optional with a normative SHOULD playbook_functionalities and functions (on command) as optional with a normative SHOULD if playbook_types are used, playbook_functionalities associated with the referenced playbook type be MUST used
     
    To support understanding and use of playbook_types and playbook_functionalities we should provide examples for the community.
     
    Separate but related: The current values for playbook_types are limited to those defined within the TC where as the community may leverage more known frameworks/taxonomies so I suggest we provide a mechanism to support other taxonomies for playbook_types.
     
    Thoughts?
     
    -Marlon
     
     


    From: cacao@lists.oasis-open.org <cacao@lists.oasis-open.org> On Behalf Of
    Vasileios Mavroeidis
    Sent: Wednesday, November 2, 2022 12:48 PM
    To: aa tt <atcyber1000@gmail.com>
    Cc: Bret Jordan <jordan.oasisopen@gmail.com>; Jason Keirstead <Jason.Keirstead@ca.ibm.com>; Dr. Desiree A Beck <dbeck@mitre.org>; duncan <duncan@sfractal.com>; cacao@lists.oasis-open.org
    Subject: Re: [cacao] [EXT] [cacao] Remaining work items


     

    CAUTION:
    This email originated from outside of DHS. DO NOT click links or open attachments unless you recognize and/or trust the sender. Contact your component SOC with questions or concerns.


     
    With regards to: " a) playbook_functionalties being required versus optional - Allan is the only one really pushing back on this. Everyone else seems to be okay with it."
     
    I was the first one against having this set of proposed properties "required". I support simplicity over complexity (Allan's points). I never heard a justification or the use case that needs them to be
    "required". Maybe I have lost episodes here, but I want to have something, so I can say, "Ah, I seeâ, and give +1. Otherwise, the proposal it self is of value.
     
    Des's compromise - a normative SHOULD - could also apply to the   "function"  property.



    -V






    On Nov 2, 2022, at 1:23 AM, aa tt <atcyber1000@gmail.com> wrote:

     


    Bret - Whether itâs 1 person or 10 people, my argument/statements on this feature and its implication on the standard are still true. I suggest we focus on the technical implications of a feature and not make it a personal attack to suggest its only me.
    My argument has merit. Donât discount it just because one person said it.

     


    I donât appreciate that tactic.


     


    Allan






    On Nov 1, 2022, at 5:05 PM, Bret Jordan <jordan.oasisopen@gmail.com> wrote:

     


    First off - 

     


    1) We have and continue to make significant progress towards a release. 


     


    2) We are working on merging that extra metadata text Allan wrote into the document. Allan, you were on the call when we decided to do that. It just has not yet been done.


     


    3) There are three major issues that are holding up the release:


    a) playbook_functionalties being required versus optional - Allan is the only one really pushing back on this. Everyone else seems to be okay with it.


    b) The attack target types are not quite right and need some work. Both Des and I have played around with them and something is just not right there. So if we are looking to release fast, then we should drop them from this version as they are not yet ready.


    c) We have had more discussion about the difference between playbook templates and executable playbooks. I think Rich's proposal today on the call can really address a lot of the concerns we have heard recently.


     


    Bret


     


    On Tue, Nov 1, 2022 at 6:42 PM aa tt < atcyber1000@gmail.com > wrote:



    In addition to Jason;s request I still havenât heard whether the metadata writeup that was written will be merged/added to the spec to help explain all the metadata properties we have in the spec.


     


    We seem to be going in the wrong direction as far as getting close to being complete on the spec. We might need a special session that everyone can make to discuss and resolve the issues. Unfortunately the meeting times havenât worked out for me and given
    that Marlon wasnât able to make the call today either then we might want to consider a specific call next week to resolve.


     


    Allan


     






    On Nov 1, 2022, at 3:08 PM, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote:

     


    Hi folks - wondering if someone can link to the detailed proposal being discussed? As Stephanie has recently left IBM I am trying to get back up to speed
    on CACAO and Allan's argument is concerning. 


     


     





    From:   cacao@lists.oasis-open.org  < cacao@lists.oasis-open.org > on behalf of aa tt < atcyber1000@gmail.com >
    Sent:  Tuesday, November 1, 2022 6:41:50 PM
    To:  Dr. Desiree A Beck < dbeck@mitre.org >
    Cc:  duncan  sfractal.com  < duncan@sfractal.com >; Bret Jordan
    < jordan.oasisopen@gmail.com >;  cacao@lists.oasis-open.org  < cacao@lists.oasis-open.org >
    Subject:  [EXTERNAL] [cacao] Re: [EXT] [cacao] Remaining work items


     






    This Message Is From an Untrusted Sender 


    You have not previously corresponded with this sender. 



    Desiree - This proposal effectively still makes the feature required. 


     


    Most, if not all, playbooks will like have commands and if we are stating that a command must also define the meta-data that describes what the command is doing then this effectively makes
    the entire feature required.


     


    Secondly, this is absolutely the worst possible scenario of all because it is equivalent to requiring every single shell command within a shell script have to have a functional description
    of what that command is doing *in addition* to the actual command itself. 


     


    This proposal results in extremely verbose playbooks with absolutely every single command having to have a descriptor in addition to the actual command itself.


     


    I am strongly opposed to this proposal as it will result in making playbooks bloated, overly cumbersome to define and effectively will result in playbooks not being defined or used at all.


     


    I suggest that all playbook metadata be optional and let the market/authors decide what is practical and reasonable to document. This is exactly the same as what occurs with other scripting
    languages. People comment on commands and scripts based on what they want to convey. They donât do it on every single command as itâs not required and would slow the development of the scripts down to a crawl.


     


     


    Allan






    On Nov 1, 2022, at 11:54 AM, Dr. Desiree A Beck < dbeck@mitre.org > wrote:

     



    Regarding the required/optional issue â as discussed on todayâs call, weâd like to propose the following compromise:


     


    - The  playbook_functionalities  property on Playbook is optional


    - The  playbook_functionalities  description includes a normative "SHOULD" (i.e., â The values for this property  SHOULD  come
    from the  playbook-function-type-ov  open vocabulary.â)


    - The  function  property on Command Data is required


     


    Dez


     




    From:   cacao@lists.oasis-open.org  < cacao@lists.oasis-open.org >  On Behalf Of  duncan  sfractal.com
    Sent:  Thursday, October 27, 2022 4:26 PM
    To:  aa tt < atcyber1000@gmail.com >; Bret Jordan < jordan.oasisopen@gmail.com >
    Cc:   cacao@lists.oasis-open.org
    Subject:  [EXT] Re: [cacao] Remaining work items




     







    On the required/ optional issue, I favor optional. I hear Allanâs arguments and I resonate with allowing new users to crawl before running. If it's that valuable the market will reward the optional feature. Remember customers can require optional features.
    It just means not everyone requires it in all cases.  Plus making required would be a breaking. Just my 2 cents. 




    Duncan




     




     







     




    iPhone, iTypo, iApologize









    From:   cacao@lists.oasis-open.org  < cacao@lists.oasis-open.org > on behalf of aa tt < atcyber1000@gmail.com >
    Sent:  Thursday, October 27, 2022 3:04 PM
    To:  Bret Jordan < jordan.oasisopen@gmail.com >
    Cc:   cacao@lists.oasis-open.org  < cacao@lists.oasis-open.org >
    Subject:  Re: [cacao] Remaining work items 



     




    Bret - 

    a) The writeup that I did on meta data remains open on adding to the spec or not as far I can tell from the last call.

    b) The proposed feature on functionality is not yet completely specified. Whether we make it required or optional, it is missing a lot of the meta-data attribution/classificaiton information that the feature allows a user to specify. So at this very moment
    the proposal is not ready to add to the spec regardless of required/optional.

    c) Adding this functionality feature brings in a lot of work upon playbook authors to define such functionality instead of focusing on the mechanics of defining a playbook in the 1st place. From my perspective it is a significant increase in the burden to do
    for every playbook and Iâm not convinced all use cases will rely on it or want to have to do it. Therefore I was willing to support its inclusion in the spec (which it HAS NOT BEEN APPROVED) provided it is optional. However, if the proponents insist that this
    feature be required then I will say that the entire feature SHOULD NOT BE ADDED to the spec at all.

    Iâm a firm supporter of proving value in the marketplace and making the barrier to entry for using playbooks very low and easy. If an optional feature (any of them) gets used reliably and well in the marketplace then the outcome is the same. 

    The idea that a spec can force people to do something they donât want to do is a fallacy. And increasing the complexity of CACAO playbooks even more than they already have is a mistake that will only result in the entire project being undermined.

    Allan

    > On Oct 27, 2022, at 10:23 AM, Bret Jordan < jordan.oasisopen@gmail.com > wrote:

    > All,

    > We have very few remaining work items that need to be addressed before we can ship the next version of the CACAO specification. We really need people to speak up and voice their opinions on the following:

    > 1) MITRE/DHS has proposed a new property to track some of the features of a playbook. Several people seem to like this and support this. However, there is some concern on if this property should be "required" or if it should be "optional". Allan strongly
    views this should be optional. MITRE/DHS really want it to be required. But I have not heard from the rest of you. 

    > Please respond to this email with your views on this subject.

    > 2) A while back Allan proposed the attack target types. I think we all generally agree that this is a good idea. However, when I have played around with it, it feels like there are some minor issues with the modeling. I have not yet been able to put my finger
    on it. To be clear I really like the idea of the Attack stuff that Allan proposed and really want it. But something just feels off with it. Has anyone else seen issues while playing around with it? 

    > Bret



    ---------------------------------------------------------------------
    To unsubscribe from this mail list, you must leave the OASIS TC that 
    generates this mail.  Follow this link to all your TCs in OASIS at:
    https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php













     







     






     














  • 2.  RE: [cacao] [EXT] [cacao] Remaining work items

    Posted 11-18-2022 16:48




    Thanks, Marlon â looks good to me too.
    Dez
     


    From: Vasileios Mavroeidis <vasileim@ifi.uio.no>
    Sent: Friday, November 18, 2022 11:24 AM
    To: Taylor, Marlon <Marlon.Taylor@cisa.dhs.gov>
    Cc: aa tt <atcyber1000@gmail.com>; Bret Jordan <jordan.oasisopen@gmail.com>; Jason Keirstead <Jason.Keirstead@ca.ibm.com>; Dr. Desiree A Beck <dbeck@mitre.org>; duncan <duncan@sfractal.com>; cacao@lists.oasis-open.org
    Subject: RE: [cacao] [EXT] [cacao] Remaining work items


     


    Looks good to me.

     


    -Vasileios 

     

     

    On Nov 18, 2022 16:47, "Taylor, Marlon" < Marlon.Taylor@cisa.dhs.gov > wrote:



    In support of adoption, as an initial early release of the specification to be evolved, I think we can have consensus around:


    playbook_types as optional with a normative SHOULD
    playbook_functionalities and functions (on command) as optional with a normative SHOULD
    if playbook_types are used, playbook_functionalities associated with the referenced playbook type be MUST used
     
    To support understanding and use of playbook_types and playbook_functionalities we should provide examples for the community.
     
    Separate but related: The current values for playbook_types are limited to those defined within the TC where as the community may leverage more known frameworks/taxonomies so I suggest we provide a mechanism to support other taxonomies for playbook_types.
     
    Thoughts?
     
    -Marlon
     
     


    From: cacao@lists.oasis-open.org < cacao@lists.oasis-open.org >
    On Behalf Of Vasileios Mavroeidis
    Sent: Wednesday, November 2, 2022 12:48 PM
    To: aa tt < atcyber1000@gmail.com >
    Cc: Bret Jordan < jordan.oasisopen@gmail.com >; Jason Keirstead < Jason.Keirstead@ca.ibm.com >; Dr. Desiree A Beck < dbeck@mitre.org >;
    duncan < duncan@sfractal.com >;
    cacao@lists.oasis-open.org
    Subject: Re: [cacao] [EXT] [cacao] Remaining work items


     

    CAUTION:
    This email originated from outside of DHS. DO NOT click links or open attachments unless you recognize and/or trust the sender. Contact your component SOC with questions or concerns.


     
    With regards to: " a) playbook_functionalties being required versus optional - Allan is the only one really pushing back on this. Everyone else seems to be okay with it."
     
    I was the first one against having this set of proposed properties "required". I support simplicity over complexity (Allan's points). I never heard a justification or the use case that needs them to be
    "required". Maybe I have lost episodes here, but I want to have something, so I can say, "Ah, I seeâ, and give +1. Otherwise, the proposal it self is of value.
     
    Des's compromise - a normative SHOULD - could also apply to the   "function"  property.

     
    -V

     


    On Nov 2, 2022, at 1:23 AM, aa tt < atcyber1000@gmail.com > wrote:

     


    Bret - Whether itâs 1 person or 10 people, my argument/statements on this feature and its implication on the standard are still true. I suggest we focus on the technical implications of a feature and not make it a personal attack to suggest its only me.
    My argument has merit. Donât discount it just because one person said it.

     


    I donât appreciate that tactic.


     


    Allan

     


    On Nov 1, 2022, at 5:05 PM, Bret Jordan < jordan.oasisopen@gmail.com > wrote:

     


    First off - 

     


    1) We have and continue to make significant progress towards a release. 


     


    2) We are working on merging that extra metadata text Allan wrote into the document. Allan, you were on the call when we decided to do that. It just has not yet been done.


     


    3) There are three major issues that are holding up the release:


    a) playbook_functionalties being required versus optional - Allan is the only one really pushing back on this. Everyone else seems to be okay with it.


    b) The attack target types are not quite right and need some work. Both Des and I have played around with them and something is just not right there. So if we are looking to release fast, then we should drop them from this version as they are not yet ready.


    c) We have had more discussion about the difference between playbook templates and executable playbooks. I think Rich's proposal today on the call can really address a lot of the concerns we have heard recently.


     


    Bret


     


    On Tue, Nov 1, 2022 at 6:42 PM aa tt < atcyber1000@gmail.com > wrote:



    In addition to Jason;s request I still havenât heard whether the metadata writeup that was written will be merged/added to the spec to help explain all the metadata properties we have in the spec.


     


    We seem to be going in the wrong direction as far as getting close to being complete on the spec. We might need a special session that everyone can make to discuss and resolve the issues. Unfortunately the meeting times havenât worked out for me and given
    that Marlon wasnât able to make the call today either then we might want to consider a specific call next week to resolve.


     


    Allan


     

     


    On Nov 1, 2022, at 3:08 PM, Jason Keirstead < Jason.Keirstead@ca.ibm.com > wrote:

     


    Hi folks - wondering if someone can link to the detailed proposal being discussed? As Stephanie has recently left IBM I am trying to get back up to speed
    on CACAO and Allan's argument is concerning. 


     


     





    From:   cacao@lists.oasis-open.org  < cacao@lists.oasis-open.org > on behalf of aa tt < atcyber1000@gmail.com >
    Sent:  Tuesday, November 1, 2022 6:41:50 PM
    To:  Dr. Desiree A Beck < dbeck@mitre.org >
    Cc:  duncan  sfractal.com  < duncan@sfractal.com >; Bret Jordan
    < jordan.oasisopen@gmail.com >;  cacao@lists.oasis-open.org  < cacao@lists.oasis-open.org >
    Subject:  [EXTERNAL] [cacao] Re: [EXT] [cacao] Remaining work items


     






    This Message Is From an Untrusted Sender 


    You have not previously corresponded with this sender. 



    Desiree - This proposal effectively still makes the feature required. 


     


    Most, if not all, playbooks will like have commands and if we are stating that a command must also define the meta-data that describes what the command is doing then this effectively makes
    the entire feature required.


     


    Secondly, this is absolutely the worst possible scenario of all because it is equivalent to requiring every single shell command within a shell script have to have a functional description
    of what that command is doing *in addition* to the actual command itself. 


     


    This proposal results in extremely verbose playbooks with absolutely every single command having to have a descriptor in addition to the actual command itself.


     


    I am strongly opposed to this proposal as it will result in making playbooks bloated, overly cumbersome to define and effectively will result in playbooks not being defined or used at all.


     


    I suggest that all playbook metadata be optional and let the market/authors decide what is practical and reasonable to document. This is exactly the same as what occurs with other scripting
    languages. People comment on commands and scripts based on what they want to convey. They donât do it on every single command as itâs not required and would slow the development of the scripts down to a crawl.


     


     


    Allan

     


    On Nov 1, 2022, at 11:54 AM, Dr. Desiree A Beck < dbeck@mitre.org > wrote:

     



    Regarding the required/optional issue â as discussed on todayâs call, weâd like to propose the following compromise:


     


    - The  playbook_functionalities  property on Playbook is optional


    - The  playbook_functionalities  description includes a normative "SHOULD" (i.e., â The values for this property  SHOULD  come
    from the  playbook-function-type-ov  open vocabulary.â)


    - The  function  property on Command Data is required


     


    Dez


     




    From:   cacao@lists.oasis-open.org  < cacao@lists.oasis-open.org >  On Behalf Of  duncan  sfractal.com
    Sent:  Thursday, October 27, 2022 4:26 PM
    To:  aa tt < atcyber1000@gmail.com >; Bret Jordan < jordan.oasisopen@gmail.com >
    Cc:   cacao@lists.oasis-open.org
    Subject:  [EXT] Re: [cacao] Remaining work items




     







    On the required/ optional issue, I favor optional. I hear Allanâs arguments and I resonate with allowing new users to crawl before running. If it's that valuable the market will reward the optional feature. Remember customers can require optional features.
    It just means not everyone requires it in all cases.  Plus making required would be a breaking. Just my 2 cents. 




    Duncan




     




     







     




    iPhone, iTypo, iApologize









    From:   cacao@lists.oasis-open.org  < cacao@lists.oasis-open.org > on behalf of aa tt < atcyber1000@gmail.com >
    Sent:  Thursday, October 27, 2022 3:04 PM
    To:  Bret Jordan < jordan.oasisopen@gmail.com >
    Cc:   cacao@lists.oasis-open.org  < cacao@lists.oasis-open.org >
    Subject:  Re: [cacao] Remaining work items 



     




    Bret - 

    a) The writeup that I did on meta data remains open on adding to the spec or not as far I can tell from the last call.

    b) The proposed feature on functionality is not yet completely specified. Whether we make it required or optional, it is missing a lot of the meta-data attribution/classificaiton information that the feature allows a user to specify. So at this very moment
    the proposal is not ready to add to the spec regardless of required/optional.

    c) Adding this functionality feature brings in a lot of work upon playbook authors to define such functionality instead of focusing on the mechanics of defining a playbook in the 1st place. From my perspective it is a significant increase in the burden to do
    for every playbook and Iâm not convinced all use cases will rely on it or want to have to do it. Therefore I was willing to support its inclusion in the spec (which it HAS NOT BEEN APPROVED) provided it is optional. However, if the proponents insist that this
    feature be required then I will say that the entire feature SHOULD NOT BE ADDED to the spec at all.

    Iâm a firm supporter of proving value in the marketplace and making the barrier to entry for using playbooks very low and easy. If an optional feature (any of them) gets used reliably and well in the marketplace then the outcome is the same. 

    The idea that a spec can force people to do something they donât want to do is a fallacy. And increasing the complexity of CACAO playbooks even more than they already have is a mistake that will only result in the entire project being undermined.

    Allan

    > On Oct 27, 2022, at 10:23 AM, Bret Jordan < jordan.oasisopen@gmail.com > wrote:

    > All,

    > We have very few remaining work items that need to be addressed before we can ship the next version of the CACAO specification. We really need people to speak up and voice their opinions on the following:

    > 1) MITRE/DHS has proposed a new property to track some of the features of a playbook. Several people seem to like this and support this. However, there is some concern on if this property should be "required" or if it should be "optional". Allan strongly
    views this should be optional. MITRE/DHS really want it to be required. But I have not heard from the rest of you. 

    > Please respond to this email with your views on this subject.

    > 2) A while back Allan proposed the attack target types. I think we all generally agree that this is a good idea. However, when I have played around with it, it feels like there are some minor issues with the modeling. I have not yet been able to put my finger
    on it. To be clear I really like the idea of the Attack stuff that Allan proposed and really want it. But something just feels off with it. Has anyone else seen issues while playing around with it? 

    > Bret



    ---------------------------------------------------------------------
    To unsubscribe from this mail list, you must leave the OASIS TC that 
    generates this mail.  Follow this link to all your TCs in OASIS at:
    https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php