Larry,
I agree, it is a reasonable statement. Here I consolidated the edits:
"SARIF represents a different strategy for common representation of the results of static analysis. The Object Management Group's Tool Output Integration Format (TOIF) is an existing standard in this space that is integrated with the OMG's software assurance
suite. TOIF normalizes and integrates the output of static analysis tools and other artifacts as evidence for software assurance.
"TOIF's strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse formats into the lowest common denominator representation without having to modify the original tools.
By contrast, SARIF aims to support the full capabilities of advanced tools, which generally requires modifying the tools to produce SARIF output natively.
"Both SARIF and TOIF solve an important problem for the organizations performing software assurance by providing a uniform and vendor-neutral way of deploying and running multiple tools on the same code base, disseminating and interpreting the combined findings,
including the reduction in the costs of training developers in how to use multiple tools and, especially, how to interpret the results from each tool.
Best regards,
Nick
On Sep 6, 2019, at 6:48 PM, Larry Golding (Myriad Consulting Inc) <
v-lgold@microsoft.com > wrote:
Nick,
Thanks for your explanation of the broad set of information that TOIF represents. That helps me understand the value of SARIF and TOIF working together.
To answer your specific question: when the convert approach is used in SARIF, it will address a common denominator between participating tools, not advanced information, would it not? :
If I understand your question correctly, the answer is no .
A SARIF converter typically consumes as much of the tool s output as SARIF can represent, including any advanced concepts that are present in the tool s output. The converter does not restrict itself to those concepts that are common among most or all tools.
If the tool s output format includes concepts that SARIF does _ not _ represent, a converter might (but is not required to) store such information in a property bag (which all SARIF objects have). For example, the SARIF result object does
not define a confidence property, so if a tool like Fortify produces a confidence value for each issue that it detects, the converter can store it in theResult.properties["confidence"].
Is that what you were asking?
I agree that the statement we have is a good one.
Larry
From: Nick Mansourov <
nick@kdmanalytics.com >
Sent: Friday, September 6, 2019 3:37 PM
To: Larry Golding (Myriad Consulting Inc) <
v-lgold@microsoft.com >
Cc: David Keaton <
dmk@dmk.com >;
sarif@lists.oasis-open.org Subject: Re: [sarif] First Draft Statement of Relationship to Similar Work
Larry,
Since SARIF is centric to the SCA space and TOIF is not, when it comes to a focused comparison restricted to the SCA space, the two distinctions lowest common denominator/advanced tools and convert / modify capture the key differences quite well.
The distinction of convert / modify may seem a minute technical detail, but it is not.
TOIF is in a broader space of software assurance, where SCA tool produces but one kind of evidence. Other kinds of evidence are produced by dynamic analysis tools, vulnerability scanning tools, OVAL tools, manual testing, etc.
Within the SCA space, TOIF normalizes the output of SCA tools so that their findings can be used as evidence for software assurance. There are other kinds of evidence that are either reported by tools other than SCA, or are not produced by tools whatsoever,
so there is no tool to modify (but there is output to convert). Therefore the TOIF ecosystem involves converters.
I think that removing the second distinction convert/modify may introduce an unwanted bias into the comparison by skipping an important aspect related to how the ecosystem of each standard is evolving.
To your point, when the convert approach is used in SARIF, it will address a common denominator between participating tools, not advanced information, would it not? When the modify existing tool or even build a new tool approach is used in TOIF, this
leads to no contradictions to its objectives.
The good news is that both SARIF and TOIF can be working together to address both the SCA space and the larger software assurance space.
I think we have a balanced statement (see modified statement below) claiming that both SARIF and TOIF address the software assurance space, where TOIF s normalizes and integrates output of static analysis tools and other artifacts as evidence for software
assurance, plus the original two points advanced / low denominator and modify / convert .
A minor edit : remove the word input in the following:
"TOIF's strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse input formats into the lowest common denominator representation without having to modify the original
tools.
(in is actually output rather than input)
What do you think ?
Best regards,
Nick
On Sep 6, 2019, at 4:14 PM, Larry Golding (Myriad Consulting Inc) <
v-lgold@microsoft.com > wrote:
My only comment is about this, referring to SARIF: "... which generally requires modifying the tools to produce SARIF output natively".
The spec describes "converters" as well as "direct producers" -- that is, converters are definitely a "thing" in SARIF -- so I suggest: "... which generally requires either modifying the tools to produce SARIF output natively, or writing a converter from the
tools's output format to SARIF."
But once you say that -- isn't the same true of TOIF? If you want TOIF, you either have to modify your tool to produce it, or (as TOIF apparently prefers) write a converter.
Larry