OASIS Static Analysis Results Interchange Format (SARIF) TC

 View Only

  • 1.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 22:37



    Larry,


    Since SARIF  is  centric to the SCA space and TOIF is not,  when it comes to a focused comparison restricted to the SCA space, the two distinctions   lowest common denominator/advanced tools and convert / modify  capture the key differences
    quite well. 


    The distinction of convert / modify may seem a minute technical detail, but it is not.
     
    TOIF is in a broader space of software assurance, where SCA tool produces but one kind of evidence. Other kinds of evidence are produced by dynamic analysis tools, vulnerability scanning tools, OVAL tools, manual testing, etc.


    Within the SCA space, TOIF normalizes the output of SCA tools so that their findings can be used as evidence for software assurance. There are other kinds of evidence that are either reported by tools other than SCA, or are not produced by tools
    whatsoever, so there is no tool to modify (but there is output to convert). Therefore the TOIF ecosystem involves converters. 
     
    I think that removing the second distinction   convert/modify may introduce an unwanted bias into the comparison by skipping an important aspect related to how the ecosystem of each standard is evolving.


    To your point, when the convert approach is used in SARIF, it will address a common denominator between participating tools, not advanced information, would it not?  When the modify existing tool or even build a new tool approach is used
    in TOIF, this leads to no contradictions to its objectives.


    The good news is that both SARIF and TOIF can be working together to address both the SCA space  and the larger software assurance space.


    I think we have a balanced statement (see modified statement below) claiming that both SARIF and TOIF address the software assurance space, where TOIF s  normalizes and integrates output of static analysis tools and other artifacts as evidence
    for software assurance, plus the original two points advanced / low denominator and modify / convert .


    A minor edit : remove the word input in the following:



    "TOIF's strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse  input  formats into the lowest common denominator representation
    without having to modify the original tools.  



    (in is actually output rather than input)


    What do you think ?


    Best regards,
    Nick





    On Sep 6, 2019, at 4:14 PM, Larry Golding (Myriad Consulting Inc) < v-lgold@microsoft.com > wrote:


    My only comment is about this, referring to SARIF: "... which generally requires modifying the tools to produce SARIF output natively".

    The spec describes "converters" as well as "direct producers" -- that is, converters are definitely a "thing" in SARIF -- so I suggest: "... which generally requires either modifying the tools to produce SARIF output natively, or writing a converter from the
    tools's output format to SARIF."

    But once you say that -- isn't the same true of TOIF? If you want TOIF, you either have to modify your tool to produce it, or (as TOIF apparently prefers) write a converter.

    Larry





  • 2.  RE: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 22:49




    Nick,
     
    Thanks for your explanation of the broad set of information that TOIF represents. That helps me understand the value of SARIF and TOIF working together.
     
    To answer your specific question: when the convert approach is used in SARIF, it will address a common denominator between participating tools, not advanced information, would it not? :
     
    If I understand your question correctly, the answer is no .
     
    A SARIF converter typically consumes as much of the tool s output as SARIF can represent, including any advanced concepts that are present in the tool s output. The converter does not restrict itself to those concepts that are common
    among most or all tools. If the tool s output format includes concepts that SARIF does _ not _ represent, a converter might (but is not required to) store such information in a property bag (which all SARIF objects have). For example, the SARIF result
    object does not define a confidence property, so if a tool like Fortify produces a confidence value for each issue that it detects, the converter can store it in theResult.properties["confidence"].
     
    Is that what you were asking?
     
    I agree that the statement we have is a good one.
     
    Larry
     


    From: Nick Mansourov <nick@kdmanalytics.com>
    Sent: Friday, September 6, 2019 3:37 PM
    To: Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>
    Cc: David Keaton <dmk@dmk.com>; sarif@lists.oasis-open.org
    Subject: Re: [sarif] First Draft Statement of Relationship to Similar Work


     
    Larry,

     


    Since SARIF  is  centric to the SCA space and TOIF is not,  when it comes to a focused comparison restricted to the SCA space, the two distinctions   lowest common denominator/advanced tools and convert / modify  capture the key differences
    quite well. 


     


    The distinction of convert / modify may seem a minute technical detail, but it is not.


     


    TOIF is in a broader space of software assurance, where SCA tool produces but one kind of evidence. Other kinds of evidence are produced by dynamic analysis tools, vulnerability scanning tools, OVAL tools, manual testing, etc.


     


    Within the SCA space, TOIF normalizes the output of SCA tools so that their findings can be used as evidence for software assurance. There are other kinds of evidence that are either reported by tools other than SCA, or are not produced
    by tools whatsoever, so there is no tool to modify (but there is output to convert). Therefore the TOIF ecosystem involves converters. 


     


    I think that removing the second distinction   convert/modify may introduce an unwanted bias into the comparison by skipping an important aspect related to how the ecosystem of each standard is evolving.


     


    To your point, when the convert approach is used in SARIF, it will address a common denominator between participating tools, not advanced information, would it not?  When the modify existing tool or even build a new tool approach
    is used in TOIF, this leads to no contradictions to its objectives.


     


    The good news is that both SARIF and TOIF can be working together to address both the SCA space  and the larger software assurance space.


     


    I think we have a balanced statement (see modified statement below) claiming that both SARIF and TOIF address the software assurance space, where TOIF s  normalizes and integrates output of static analysis tools and other artifacts as
    evidence for software assurance, plus the original two points advanced / low denominator and modify / convert .


     


    A minor edit : remove the word input in the following:


     



    "TOIF's strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse  input  formats into the lowest common denominator representation without having to modify
    the original tools.  



     


    (in is actually output rather than input)


     


    What do you think ?


     


    Best regards,


    Nick


     


     



    On Sep 6, 2019, at 4:14 PM, Larry Golding (Myriad Consulting Inc) < v-lgold@microsoft.com > wrote:

     


    My only comment is about this, referring to SARIF: "... which generally requires modifying the tools to produce SARIF output natively".

    The spec describes "converters" as well as "direct producers" -- that is, converters are definitely a "thing" in SARIF -- so I suggest: "... which generally requires either modifying the tools to produce SARIF output natively, or writing a converter from the
    tools's output format to SARIF."

    But once you say that -- isn't the same true of TOIF? If you want TOIF, you either have to modify your tool to produce it, or (as TOIF apparently prefers) write a converter.

    Larry



    Original Message


  • 3.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 22:57
    In case anybody else had trouble understanding, I'll save you some time. SCA means Static Code Analysis. I had never heard it referred to with that acronym, even though I used to work at CERT, so I had to look it up. On 2019-09-06 16:36, Nick Mansourov wrote: A minor edit : remove the word input in the following: "TOIF's strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse input formats into the lowest common denominator representation without having to modify the original tools. (in is actually output rather than input) What do you think ? The tool's output is TOIF's input, which is what I was thinking. I agree that it works better to delete the word input, though, since it's really both output and input. TOIF normalizes the output of static analysis tools so that it can be used as evidence for digital certification of software. TOIF normalizes and integrates the output of static analysis tools and other artifacts as evidence for software assurance. Yes, I like this new version of the sentence better. Thanks. David


Related Content