OASIS Static Analysis Results Interchange Format (SARIF) TC

Back to discussions
Expand all | Collapse all

First Draft Statement of Relationship to Similar Work

  • 1.  First Draft Statement of Relationship to Similar Work

    Posted 09-04-2019 15:21
    To promote SARIF to Candidate OASIS Standard, we need a statement of relationship to similar work. I'd like to propose the following as a first draft. Comments and improvements are welcome. "SARIF represents an evolution in strategy for common representation of the results of static analysis. The previous generation includes the Object Management Group's Tool Output Integration Format (TOIF) standard, and software products such as Code DX and Thread Fix. Their strategy involves creating adapters from various tools to the reporting format, and as such, they are focused on integrating the diverse input formats into the lowest common denominator representation without having to modify the original tools. "By contrast, SARIF aims to support the full capabilities of advanced tools, which generally requires modifying the tools to produce SARIF output natively." David


  • 2.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-04-2019 15:26
    Actually, we are only required to mention standards, so maybe it would be better to delete the product references. Anyway, I'm still interested in input from others. David On 2019-09-04 09:20, David Keaton wrote: To promote SARIF to Candidate OASIS Standard, we need a statement of relationship to similar work. I'd like to propose the following as a first draft. Comments and improvements are welcome. "SARIF represents an evolution in strategy for common representation of the results of static analysis. The previous generation includes the Object Management Group's Tool Output Integration Format (TOIF) standard, and software products such as Code DX and Thread Fix. Their strategy involves creating adapters from various tools to the reporting format, and as such, they are focused on integrating the diverse input formats into the lowest common denominator representation without having to modify the original tools. "By contrast, SARIF aims to support the full capabilities of advanced tools, which generally requires modifying the tools to produce SARIF output natively." David --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php


  • 3.  RE: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-04-2019 17:53
    I'd also avoid words that imply SARIF is superior, such as "an evolution of strategy", and that existing tools are obsolete, such as "the previous generation". Perhaps: "a different strategy" and "existing standards include".
    Original Message


  • 4.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-04-2019 18:19
    Good point, thanks. I agree. David On 2019-09-04 11:52, Larry Golding (Myriad Consulting Inc) wrote: I'd also avoid words that imply SARIF is superior, such as "an evolution of strategy", and that existing tools are obsolete, such as "the previous generation". Perhaps: "a different strategy" and "existing standards include".
    Original Message


  • 5.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 15:23
    Here's what we have so far. This is sufficient to fulfill our requirement for a standards relationship statement, though comments are still welcome. "SARIF represents a different strategy for common representation of the results of static analysis. The Object Management Group's Tool Output Integration Format (TOIF) is an existing standard in this space. Its strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse input formats into the lowest common denominator representation without having to modify the original tools. "By contrast, SARIF aims to support the full capabilities of advanced tools, which generally requires modifying the tools to produce SARIF output natively." David P.S. Thanks to Nick Mansourov for the insight about TOIF being designed to produce the lowest common denominator output for compatibility's sake, as shown in this submission of his to the SARIF TC. https://www.oasis-open.org/apps/org/workgroup/sarif/download.php/62623/Alignment_SARIF_TOIF.pdf


  • 6.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 16:55
    Hello all, I like this revision. May I suggest the following additions (inline): > On Sep 6, 2019, at 11:22 AM, David Keaton <dmk@dmk.com> wrote: > > Here's what we have so far. This is sufficient to fulfill our requirement for a standards relationship statement, though comments are still welcome. > > "SARIF represents a different strategy for common representation of the results of static analysis. The Object Management Group's Tool Output Integration Format (TOIF) is an existing standard in this space. Its strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse input formats into the lowest common denominator representation without having to modify the original tools. > TOIF solves an important problem for the organizations performing software assurance by providing a uniform and vendor-neutral way of deploying and running multiple tools on the same code base, disseminating and interpreting the combined findings, including the reduction in the costs of training developers in how to use multiple tools and, especially, how to interpret the results from each tool. TOIF is integrated with several other OMG standards related to software assurance. > "By contrast, SARIF aims to support the full capabilities of advanced tools, which generally requires modifying the tools to produce SARIF output natively. Both SARIF and TOIF encourage third-party developers to build new capabilities for software assurance in a larger ecosystem of tools and services. Both specifications are aligned at their core concepts and a roadmap for interoperability has been defined. > > David > > P.S. Thanks to Nick Mansourov for the insight about TOIF being designed to produce the lowest common denominator output for compatibility's sake, as shown in this submission of his to the SARIF TC. > > https://www.oasis-open.org/apps/org/workgroup/sarif/download.php/62623/Alignment_SARIF_TOIF.pdf > > --------------------------------------------------------------------- > To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php


  • 7.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 18:05
    Nick, Thanks. This is actually supposed to be a very brief statement that fits into the form for requesting that SARIF be promoted to Candidate OASIS Standard. It is primarily about SARIF's relationship to other standards. To illustrate the level of explanation that we are looking for, here is the summary of what SARIF is, which is a separate statement for the same form that we discussed earlier. "Static Analysis Results Interchange Format (SARIF) is a standard output format for static analysis tools. A static analysis tool is a program that examines programming artifacts in order to detect problems, without executing the program. A standard output format allows results to be combined across runs of the same tool, and across runs of tools from multiple vendors, to get a more complete picture of the aspects of a program that need improvement." To match this level of explanation, and to keep the statement of relationship to similar work brief and more focused on SARIF, I'd like to suggest the following modification. "SARIF represents a different strategy for common representation of the results of static analysis. The Object Management Group's Tool Output Integration Format (TOIF) is an existing standard in this space that is integrated with the OMG's software assurance suite. TOIF's strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse input formats into the lowest common denominator representation without having to modify the original tools. "By contrast, SARIF aims to support the full capabilities of advanced tools, which generally requires modifying the tools to produce SARIF output natively. "Both SARIF and TOIF solve an important problem for the organizations performing software assurance by providing a uniform and vendor-neutral way of deploying and running multiple tools on the same code base, disseminating and interpreting the combined findings, including the reduction in the costs of training developers in how to use multiple tools and, especially, how to interpret the results from each tool." David


  • 8.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 19:01
    David, OK, we could apply the characterization that I have originally used for TOIF for both TOIF and SARIF as you suggest. In this case I would like to add one more sentence to better characterize TOIF s objective (see below). I can promote this write-up with the OMG community. Best regards, Nick > On Sep 6, 2019, at 2:04 PM, David Keaton <dmk@dmk.com> wrote: > > Nick, > > Thanks. This is actually supposed to be a very brief statement that fits into the form for requesting that SARIF be promoted to Candidate OASIS Standard. It is primarily about SARIF's relationship to other standards. To illustrate the level of explanation that we are looking for, here is the summary of what SARIF is, which is a separate statement for the same form that we discussed earlier. > > "Static Analysis Results Interchange Format (SARIF) is a standard output format for static analysis tools. A static analysis tool is a program that examines programming artifacts in order to detect problems, without executing the program. A standard output format allows results to be combined across runs of the same tool, and across runs of tools from multiple vendors, to get a more complete picture of the aspects of a program that need improvement." > > To match this level of explanation, and to keep the statement of relationship to similar work brief and more focused on SARIF, I'd like to suggest the following modification. > > "SARIF represents a different strategy for common representation of the results of static analysis. The Object Management Group's Tool Output Integration Format (TOIF) is an existing standard in this space that is integrated with the OMG's software assurance suite. TOIF's strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse input formats into the lowest common denominator representation without having to modify the original tools. TOIF normalizes the output of static analysis tools so that it can be used as evidence for digital certification of software. > > "By contrast, SARIF aims to support the full capabilities of advanced tools, which generally requires modifying the tools to produce SARIF output natively. > > "Both SARIF and TOIF solve an important problem for the organizations performing software assurance by providing a uniform and vendor-neutral way of deploying and running multiple tools on the same code base, disseminating and interpreting the combined findings, including the reduction in the costs of training developers in how to use multiple tools and, especially, how to interpret the results from each tool." > > David


  • 9.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 19:10
    Nick, Thanks. How about one small change to keep the two strategies together so that the "By contrast . . ." still makes the most sense. "SARIF represents a different strategy for common representation of the results of static analysis. The Object Management Group's Tool Output Integration Format (TOIF) is an existing standard in this space that is integrated with the OMG's software assurance suite. TOIF normalizes the output of static analysis tools so that it can be used as evidence for digital certification of software. "TOIF's strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse input formats into the lowest common denominator representation without having to modify the original tools. By contrast, SARIF aims to support the full capabilities of advanced tools, which generally requires modifying the tools to produce SARIF output natively. "Both SARIF and TOIF solve an important problem for the organizations performing software assurance by providing a uniform and vendor-neutral way of deploying and running multiple tools on the same code base, disseminating and interpreting the combined findings, including the reduction in the costs of training developers in how to use multiple tools and, especially, how to interpret the results from each tool." David


  • 10.  RE: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 20:14
    My only comment is about this, referring to SARIF: "... which generally requires modifying the tools to produce SARIF output natively". The spec describes "converters" as well as "direct producers" -- that is, converters are definitely a "thing" in SARIF -- so I suggest: "... which generally requires either modifying the tools to produce SARIF output natively, or writing a converter from the tools's output format to SARIF." But once you say that -- isn't the same true of TOIF? If you want TOIF, you either have to modify your tool to produce it, or (as TOIF apparently prefers) write a converter. Larry
    Original Message


  • 11.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 20:21
    Larry, Maybe we could change "generally" to "often." The point is that SARIF provides access to features that often can't be gleaned by converters, in contrast to TOIF's lowest common denominator approach. David On 2019-09-06 14:14, Larry Golding (Myriad Consulting Inc) wrote: My only comment is about this, referring to SARIF: "... which generally requires modifying the tools to produce SARIF output natively". The spec describes "converters" as well as "direct producers" -- that is, converters are definitely a "thing" in SARIF -- so I suggest: "... which generally requires either modifying the tools to produce SARIF output natively, or writing a converter from the tools's output format to SARIF." But once you say that -- isn't the same true of TOIF? If you want TOIF, you either have to modify your tool to produce it, or (as TOIF apparently prefers) write a converter. Larry
    Original Message


  • 12.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 20:27
    Larry, The other thing to remember is that before SARIF, most conversion strategies operated on the human-readable output of tools, which missed a lot of the internals that can help characterize a run. David On 2019-09-06 14:20, David Keaton wrote: Larry, Maybe we could change "generally" to "often." The point is that SARIF provides access to features that often can't be gleaned by converters, in contrast to TOIF's lowest common denominator approach. David On 2019-09-06 14:14, Larry Golding (Myriad Consulting Inc) wrote: My only comment is about this, referring to SARIF: "... which generally requires modifying the tools to produce SARIF output natively". The spec describes "converters" as well as "direct producers" -- that is, converters are definitely a "thing" in SARIF -- so I suggest: "... which generally requires either modifying the tools to produce SARIF output natively, or writing a converter from the tools's output format to SARIF." But once you say that -- isn't the same true of TOIF? If you want TOIF, you either have to modify your tool to produce it, or (as TOIF apparently prefers) write a converter. Larry
    Original Message


  • 13.  RE: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 20:37
    Responding to your comment of 2019-09-06 14:20, "... SARIF provides access to features that often can't be gleaned by converters": generally, the things a converter doesn't know aren't related to the feature-richness of the tool's output format. They are more bookkeeping details, like: - Was the file system on which the tool was run case-sensitive? - Is the tool's 3-component version number actually a semantic version? - Can I safely remove a '..' segment from a URL, or is there a symbolic link in play? - Is the file in which the tool detected a result also the file that the tool was instructed to scan (the "analysis target")? Responding to your follow-up comment, "... the human-readable output of tools, [misses] a lot of the internals that can help characterize a run": I agree that if a tool vendor decides to produces SARIF natively, they has the opportunity to populate more information than their native output format provides. And I suppose you could argue that since TOIF aspires to be a least-common-denominator interchange format, it is less likely that a tool vendor who decided to emit TOIF natively would all of a sudden discover opportunities to provide richer output. Is that the point you're making? At this point, having made my contribution, which was just to remind you that "converters are a thing", I'm fine with changing "generally" to "often" and calling it a day. Larry
    Original Message


  • 14.  Re: [sarif] First Draft Statement of Relationship to Similar Work

    Posted 09-06-2019 20:47
    On 2019-09-06 14:37, Larry Golding (Myriad Consulting Inc) wrote: I suppose you could argue that since TOIF aspires to be a least-common-denominator interchange format, it is less likely that a tool vendor who decided to emit TOIF natively would all of a sudden discover opportunities to provide richer output. Is that the point you're making? Right. Also, remember that the statement has to show the rest of OASIS an answer to the question "Why does SARIF exist when there is TOIF?" otherwise there is no point in their voting to make it an OASIS Standard. The whole of OASIS votes on that, not just the SARIF TC. At this point, having made my contribution, which was just to remind you that "converters are a thing", I'm fine with changing "generally" to "often" and calling it a day. Sounds good, thanks. David


Related Content